Some thoughts on what the implementation should look like
I want a single process to manage the queue, but I want it to be able to fork before performing any interesting work. This should allow it to scale well but not get involved it too much hairy IPC.
Event loop
The main thread (and probably all sub threads) will run around an event loop, listening on sockets and dealing with what it gets.- There will be some sockets on which it is listening for connections. When a connection is made, it will either fork to handle it after setting up a pipe for return information, or will handle it directly.
- There will be some pipe on which it is waiting for information form a child. These will be filenames which will be added to the queue for future processing. The filename might also have a time which is the next time to process the file, or a indication that the file should be deleted.
- There may be sockets on which it is serving SMTP or LMTP. For each of these it must maintain the state of a server and have a control file into which information is being added. It will need to know how the socket should authenticate, and whether authentication has happened.
- There may be sockets on which it is acting as an SMTP or LMTP client. These will need to be state-machine based clients sending authentication and addresses and dealing with the result status of each address by appending information to the control file and possibly updating internal state.
As messages appear that need processing, those messages will either be queued for an outgoing socket, or a child will be forked to handle each. The queue manager may keep a number of outgoing sockets in an already-authenticated state (if the destination handler allows that) to reduce latency. This would particularly be helpful for destinations that were programs that needed to be started by the queue manager
server configuration
The queue manager acts as an SMTP/LMTP server listening either on a unix domain socket or a TCP socket. For each listening address we need to specify the address, the authentication method, the authorisation method, and the destination.
For Unix domain sockets, SCM_CREDENTIALS