runtime: support for daemonize
gopherbot opened this issue · comments
by azummo-google@towertech.it:
(Take this as LongTerm or Thinking)
It would be fine to have a fork and daemonize calls,
something like this:
func Daemonize()
{
os.Stdin.Close();
os.Stdout.Close();
os.Stderr.Close();
pid := Fork();
// In parent
if pid != 0 {
os.Exit(0);
}
// In child
syscall.Setsid()
....
}
Unfortunately, this is a bit harder than it looks. On most systems, threads and daemonize do not play well with each other.
Owner changed to r...@golang.org.
Status changed to LongTerm.
Comment 2 by azummo-google@towertech.it:
I'm sure it is! Here (linux/386) I'm trying daemonizing early in main() and launching goroutine safterwards. Seems working nicely.
Comment 4 by graycardinalster:
Full analog of "daemon(3)"
import (
"syscall"
"os"
)
func daemon (nochdir, noclose int) int {
var ret uintptr
var err uintptr
ret,_,err = syscall.Syscall(syscall.SYS_FORK, 0, 0, 0)
if err != 0 { return -1 }
switch (ret) {
case 0:
break
default:
os.Exit (0)
}
if syscall.Setsid () == -1 { return -1 }
if (nochdir == 0) { os.Chdir("/") }
if noclose == 0 {
f, e := os.Open ("/dev/null", os.O_RDWR, 0)
if e == nil {
fd := f.Fd ()
syscall.Dup2 (fd, os.Stdin.Fd ())
syscall.Dup2 (fd, os.Stdout.Fd ())
syscall.Dup2 (fd, os.Stderr.Fd ())
}
}
return 0
}While it might be useful to have a Fork() function for other purposes, I'd suggest omitting and discouraging any sort of Daemon()-like feature. It's usually a mistake for a daemon to fork itself. Self-daemonizing makes life unnecessarily hard for anyone who wants to monitor the daemon process.
I've run into a number of cases where, say, a given Linux distribution doesn't provide a useful/secure/correct daemonization wrapper utility, if any at all. In this case, self-daemonization, which is the defacto method, is assumed (and a good self-daemonizing program will have an option to leave a pidfile and logfiles, which will make it easy to monitor the process). Since daemonization really only occurs in the wild during initialization, it really ought to be called from an init function. In order to make fork safe for daemonization, and seeing that the spec has now changed (goroutines now *can* run during initialization), all we need to do is any one of three things: 1) Disallow creation of os threads during initialization, where go's concurrency support is only for concurrency, not parallelization, as Rob Pike might say -- all goroutine switching during init would probably need to be cooperative (through blocking or runtime.Gosched), which may not be possible to accomplish in the current runtime. 2) Create something like a runtime.SuspendAll() function, upon returning, guarantees that all other threads are muxed off of os threads (which are destroyed) and the calling goroutine is run alone until a complimentary function, like a runtime.ResumeAll() is called. This is not the same as GOMAXPROCS, which doesn't restrict the number of threads used by the runtime (this option would need to account for those as well). 3) Amend the published spec to require that unrelated dependencies initialize in code-order (this would also mean that gofmt wouldn't be allowed to reorder imports), or at least that independent import groups occur in source order. So at the minimum, the following should require that daemon initialize fully before threadspawner: import "daemon" // daemon imports nothing import ( "fmt" "os" "threadspawner" // threadspawner imports nothing ) (Binaries produced from 6g and friends appear to do this in either undefined or reverse order). Knowing nothing about the toolchain/runtime internals, #3 is probably the simplest to implement -- if you want to argue that the compiler be allowed to optimize for speed (putting goroutines-spawning initializers before non-concurrent initializers), keep in mind that initialization is not a critical place for speed (in a long running program, initialization time counts for nothing, and in short running programs, heavy lifting is usually *not* done in initialization).
Comment 14 by cw@f00f.org:
We can make daemonize work 'as expected' with a few restrictions. These restrictions won't apply to most people and almost never if called in or near program start-up. These restrictions aren't unique to Go, but we can detect and error on them rather than silently ignore them (as is sometimes the case elsewhere).
Special casing a standard daemon package actually sounds like the most intuitive thing -- thanks Russ! In any case, programmers who roll their own almost always do the wrong thing (I think even the C daemon call doesn't handle it even close to securely), and we shouldn't make them worry about implementation details.
Special casing a standard daemon package actually sounds like the most intuitive thing -- thanks Russ! In any case, programmers who roll their own almost always do the wrong thing (I think even the C daemon call doesn't handle it even close to securely), and we shouldn't make them worry about implementation details.
Would your changes allow one to have multiple processes accepting socket connections for single listening socket like C does*? basically: proc1: sock.listen proc1: fork N children procN: s = sock.accept procN: do work on s * on Linux this is possible not sure about other OSes
Comment 23 by cw@f00f.org:
Or you could use SCM_RIGHTS after the fact.
Comment 26 by matthias.benkmann:
How about this
type MainFunc func(...interface{})
// Forks a child process and executes main(args...) within that process.
// Returns the process id of the child.
//
// Within the child process, a new Go runtime is initialized from scratch, as
// if the program had started its own binary via os.StartProcess(). However as
// the last step instead of starting main.main(), the child process will start
// the main function passed to Fork().
//
// The parameters args are passed via encoding/gob with the corresponding
// restrictions.
//
// Open file descriptors, os.Args and the environment are inherited from
// the parent process, NOT reset to the values they had at the start of
// the program.
//
// Note: Fork() does not actually re-execute the program's binary.
// The program's binary need not be executable for purposes of Fork()
// Fork() will succeed even if the process no longer has the necessary
// privileges to execute the binary or if it has been (re)moved.
func Fork(main MainFunc, args... interface{}) (int, error) {
...
}
This Fork() has well-defined semantics and can be used for daemonizing as well as many
other uses of fork().
Implementing the ability to set up a new runtime environment without accessing the
binary shouldn't be too tough. The compiler might have to be changed to keep an
additional read-only copy of initial values for certain structures around. But that
doesn't require changing the language spec.You can already detach from the controlling terminal by using StartProcess and setting stdin,stdout, and stderr to nil (Only tested on ubuntu 12.04 bash in a gnome-terminal). However we would still need a fork() approach if you wanted to drop privileges before detaching. Rory McGuire Email: rjmcguire@gmail.com Mobile: (+27)(0) 82 856 3646
Comment 29 by sergey.yarmonov:
You can try package https://github.com/sevlyar/go-daemon
I found those daemon packages that have emerged too complicated, here is my solution:
package main
import (
"log"
"os"
"os/exec"
"syscall"
"time"
)
func main() {
// check command line flags, configuration etc.
// short delay to avoid race condition between os.StartProcess and os.Exit
// can be omitted if the work done above amounts to a sufficient delay
time.Sleep(1 * time.Second)
if os.Getppid() != 1 {
// I am the parent, spawn child to run as daemon
binary, err := exec.LookPath(os.Args[0])
if err != nil {
log.Fatalln("Failed to lookup binary:", err)
}
_, err = os.StartProcess(binary, os.Args, &os.ProcAttr{Dir: "", Env: nil,
Files: []*os.File{os.Stdin, os.Stdout, os.Stderr}, Sys: nil})
if err != nil {
log.Fatalln("Failed to start process:", err)
}
os.Exit(0)
} else {
// I am the child, i.e. the daemon, start new session and detach from terminal
_, err := syscall.Setsid()
if err != nil {
log.Fatalln("Failed to create new session:", err)
}
file, err := os.OpenFile("/dev/null", os.O_RDWR, 0)
if err != nil {
log.Fatalln("Failed to open /dev/null:", err)
}
syscall.Dup2(int(file.Fd()), int(os.Stdin.Fd()))
syscall.Dup2(int(file.Fd()), int(os.Stdout.Fd()))
syscall.Dup2(int(file.Fd()), int(os.Stderr.Fd()))
file.Close()
}
// daemon business logic starts here
}
Your race-avoidance code still forms a race condition
This should eliminate the race condition. Instead of the sleep, do:
err := syscall.FcntlFlock(os.Stdout.Fd(), syscall.F_SETLKW, &syscall.Flock_t{
Type: syscall.F_WRLCK, Whence: 0, Start: 0, Len: 0 })
if err != nil {
log.Fatalln("Failed to lock stdout:", err)
}
Yeah, I think it's time we finally closed this. The answer is init systems like systemd, launchd, daemontools, supervisor, runit, Kubernetes, heroku, Borg, etc etc.
Don't always need systemd, launchd, daemontools, supervisor, runit, Kubernetes, heroku, Borg, etc etc. when you have the shell's & operator.