Write some more docs

1 files changed, 63 insertions, 0 deletions

diff --git a/docs/stator.rst b/docs/stator.rst
new file mode 100644
index 0000000..0ddd05c
--- /dev/null
+++ b/docs/stator.rst
@@ -0,0 +1,63 @@
+Stator
+======
+
+Takahē's background task system is called Stator, and rather than being a
+transitional task queue, it is instead a *reconciliation loop* system; the
+workers look for objects that could have actions taken, try to take them, and
+update them if successful.
+
+As someone running Takahē, the most important aspects of this are:
+
+* You have to run at least one Stator worker to make things like follows,
+  posting, and timelines work.
+
+* You can run as many workers as you want; there is a locking system to ensure
+  they can coexist.
+
+* You can get away without running any workers for a few minutes; the server
+  will continue to accept posts and follows from other servers, and will
+  process them when a worker comes back up.
+
+* There is no separate queue to run, flush or replay; it is all stored in the
+  main database.
+
+* If all your workers die, just restart them, and within a few minutes the
+  existing locks will time out and the system will recover itself and process
+  everything that's pending.
+
+You run a worker via the command ``manage.py runstator``. It will run forever
+until it is killed; send SIGINT (Ctrl-C) to it once to have it enter graceful
+shutdown, and a second time to force exiting immediately.
+
+
+Technical Details
+-----------------
+
+Each object managed by Stator has a set of extra columns:
+
+* ``state``, the name of a state in a state machine
+* ``state_ready``, a boolean saying if it's ready to have a transition tried
+* ``state_changed``, when it entered into its current state
+* ``state_attempted``, when a transition was last attempted
+* ``state_locked_until``, when the entry is locked by a worker until
+
+They also have an associated state machine which is a subclass of
+``stator.graph.StateGraph``, which will define a series of states, the
+possible transitions between them, and handlers that run for each state to see
+if a transition is possible.
+
+An object becoming ready for execution happens first:
+
+* If it's just entered into a new state, or just created, it is marked ready.
+* If ``state_attempted`` is far enough in the past (based on the ``try_interval``
+  of the current state), a small scheduling loop marks it as ready.
+
+Then, in the main fast loop of the worker, it:
+
+* Selects an item with ``state_ready`` that is in a state it can handle (some
+  states are "externally progressed" and will not have handlers run)
+* Fires up a coroutine for that handler and lets it run
+* When that coroutine exits, sees if it returned a new state name and if so,
+  transitions the object to that state.
+* If that coroutine errors or exits with ``None`` as a return value, it marks
+  down the attempt and leaves the object to be rescheduled after its ``try_interval``.