RetroArch/network/netplay/README

This is RetroArch's Netplay code. RetroArch Netplay allows a second player to
be connected via the Internet/LAN, rather than local at the same computer. Netplay
in RetroArch is guaranteed* to work with perfect synchronization given a few
minor constraints:

(1) The core is deterministic,
(2) The only input devices the core interacts with are the joypad and analog sticks, and
(3) Both the core and the loaded content are identical on host and client.

Furthermore, if the core supports serialization (save states), Netplay allows
for latency and clock drift, providing both host and client with a smooth
experience, as well as the option of more than two players.

Netplay in RetroArch works by expecting input to come delayed from the network,
then rewinding and re-playing with the delayed input to get a consistent state.
So long as both sides agree on which frame is which, it should be impossible
for them to become de-synced, since each input event always happens at the
correct frame.

In terms of the implementation, Netplay is in effect a state buffer
(implemented as a ring of buffers) and some pre- and post-frame behaviors.

Within the state buffers, there are three locations: self, other and unread.
Each refers to a frame, and a state buffer corresponding to that frame. The
state buffer contains the savestate for the frame, and the input from both the
local and remote players.

Self is where the emulator believes itself to be, which may be ahead or behind
of what it's read from the peer. Generally speaking, self progresses at 1 frame
per frame, but both fast-forwarding and stalling occur and are described later.

Other is where it was most recently in perfect sync: i.e., other-1 is the last
frame from which both local and remote input have been actioned. As such, other
is always less than or equal to both self and read. Since the state buffer is a
ring, other is the first frame that it's unsafe to overwrite.

Unread is the first frame at which not all players' data has been read, which
can be slightly ahead of other since it can't always immediately act upon new
data.

In general, other ≤ unread and other ≤ self. In all likelihood, unread ≤ self,
but it is both possible and supported for the remote host to get ahead of the
local host.

There is an additional location, run, used when input latency is desired. In
that case, self points to where input is being read, and run points to the
frame actually being executed. Run is purely local.

The server has a slightly more complicated job as it can handle multiple
clients, however it is not vastly more complicated: For each connection which
is playing (i.e., has a controller), it maintains a per-player unread frame,
and the global unread frame is the earliest of each player unread frame. The
server forwards input data: When input data is received from an earlier frame
than the server's current frame, it forwards it immediately. Otherwise, it
forwards it when the frame is reached. i.e., during frame n, the server may
send its own and any number of other players' data for frame n, but will never
send frame n+1. This is because the server's clock is the arbiter of all
synchronization-related events, players joining and parting,
and saving/loading states.

Pre-frame, Netplay serializes the core's state, polls for local input, and
polls for input from the network. If the input from the network is too far
behind (i.e., unread is too far behind self), it stalls to allow the other side
to catch up. To assure that this stalling does not block the UI thread, it is
implemented similarly to pausing, rather than by blocking on the socket.

If input has not been received for the other side up to the current frame (the
usual case), the remote input is simulated in a simplistic manner.  Each
frame's local serialized state and simulated or real input goes into the frame
buffers.

During the frame of execution, when the core requests input, it receives the
input from the state buffer, both local and real or simulated remote.

Post-frame, it checks whether it's read more than it's actioned, i.e. if read >
other and self > other. If so, it first checks whether its simulated remote
data was correct. If it was, it simply moves other up. If not, it rewinds to
other (by loading the serialized state there) and runs the core in replay mode
with the real data up to the least of self and read, then sets other to that.
To avoid latency building up, if the input from the network is too far ahead
(i.e., unread is too far ahead of self), the frame limiter is momentarily
disabled to catch up. Note that since network latency is expected, the normal
case is the opposite: unread is behind self.

When in Netplay mode, the callback for receiving input is replaced by
input_state_net. It is the role of input_state_net to combine the true local
input (whether live or replay) with the remote input (whether true or
simulated).

Some thoughts about "frame counts": The frame counters act like indexes into a
0-indexed array; i.e., they refer to the first unactioned frame. So, when
read_frame_count is 23, we've read 23 frames, but the last frame we read is
frame 22. With self_frame_count it's slightly more complicated, since there are
two relevant actions: Reading the data and emulating with the data. The frame
count is only incremented after the latter, so there is a period of time during
which we've actually read self_frame_count+1 frames of local input.

Clients may come and go, and may start or stop playing even as they're
connected. A client that is not playing is said to be “spectating”: It receives
all the same data but sends none. A client may switch from spectating to
playing by sending the appropriate request, at which point it is allotted a
player number (see the SPECTATE, PLAY and MODE commands below).

The server may also be in spectator mode, but as the server never sends data
early (i.e., it only forwards data on the frame it's reached), it must also
inform all clients of its own current frame even if it has no input. The
NOINPUT command is provided for that purpose.

Each client has a client number, and the server is always client number 0.
Client numbers are currently limited to 0-31, as they're used in 32-bit
bitmaps.

The handshake procedure (this part is done by both server and client):
1) Send connection header
2) Receive and verify connection header
3) Send nickname
4) Receive nickname

For the client:
5) Send PASSWORD if applicable
4) Receive INFO
5) Send INFO
6) Receive SYNC

For the server:
5) Receive PASSWORD if applicable
6) Send INFO
7) Receive INFO
8) Send SYNC

* Guarantee not actually a guarantee.

Netplay's command format

Netplay commands consist of a 32-bit command identifier, followed by a 32-bit
payload size, both in network byte order, followed by a payload. The command
identifiers are listed in netplay_private.h. The commands are described below. Unless
specified otherwise, all payload values are in network byte order.

Command: ACK
Payload: None
Description:
    Acknowledgement. Not used.

Command: NAK
Payload: None
Description:
    Negative Acknowledgement. If received, the connection is terminated. Sent
    whenever a command is malformed or otherwise not understood.

Command: DISCONNECT
Payload: None
Description:
    Gracefully disconnect. Not used.

Command: INPUT
Payload:
    {
       frame number: uint32
       client number: uint32
       input data: variable
    }
Description:
    Input state for each frame. Netplay must send an INPUT command for every
    frame in order to function at all. When sent from the server (client 0),
    INPUT is a synchronization point: No synchronization events from the given
    frame may arrive after the server's input for the frame. The actual size of
    the input data is variable, but must match the expected size for each of
    the devices the given client controls.

Command: NOINPUT
Payload:
    {
       frame number: uint32
    }
Description:
    Sent by the server to indicate a frame has passed when the server is not
    otherwise sending data.

Command: NICK
Payload:
    {
       nickname: char[32]
    }
Description:
    Send nickname. Mandatory handshake command.

Command: PASSWORD
Payload:
    {
       password hash: char[64]
    }
Description:
    Send hashed password to server. Mandatory handshake command for clients if
    the server demands a password.

Command: INFO
Payload
    {
       content CRC: uint32
       core name: char[32]
       core version: char[32]
    }
Description:
    Send core/content info. Mandatory handshake command. Sent by server first,
    then by client, and must match. Server may send INFO with no payload, in
    which case the client sends its own info and expects the server to load the
    appropriate core and content then send a new INFO command. If mutual
    agreement cannot be achieved, the correct solution is to simply disconnect.

Command: SYNC
Payload:
    {
       frame number: uint32
       paused?: 1 bit
       client number: 31 bits
       controller devices: uint32[16]
       share modes: uint8[16]
       controller-client mapping: uint32[16]
       client nick: char[32]
       sram: variable
    }
Description:
    Initial state synchronization. Mandatory handshake command from server to
    client only. Controller devices are the devices plugged into each controller port,
    and 16 is really MAX_USERS.
    Client is forced to have a different nick if multiple clients have the same
    nick.

Command: SPECTATE
Payload: None
Description:
    Request to enter spectate mode. The client should immediately consider
    itself to be in spectator mode and send no further input.

Command: PLAY
Payload:
    {
       as slave?: 1 bit
       reserved: 7 bits
       preferred share mode: 8 bits
       requested device(s): 16 bits
    }
Description:
    Request to enter player mode. The client must wait for a MODE command
    before sending input. Server may refuse or force slave connections, so the
    request is not necessarily honored. The server may choose to override the
    share mode, and if no devices are explicitly requested, the server may
    choose how to assign. The default handling of the share mode is to always
    accept it if the client is the first to get the device. If the share mode
    requested is don't-care, then a default share mode is chosen. The default
    device selection depends on the number of devices available:  If there is
    only one device, then the default is to share that device. If there are
    more than one, then the default is to refuse PLAY requests when all are
    assigned.

Command: MODE
Payload:
    {
       frame number: uint32
       you: 1 bit
       playing: 1 bit
       slave: 1 bit
       reserved: 13 bits
       client number: uint16
       device bitmap: uint32
       share modes: uint8[16]
       nick: char[32]
    }
Description:
    Inform of a connection mode change (possibly of the receiving client). Only
    server-to-client. Frame number is the first frame in which player data is
    expected, or the first frame in which player data is not expected. In the
    case of new players the frame number must be later than the last frame of
    the server's own input that has been sent, and in the case of leaving
    players the frame number must be later than the last frame of the relevant
    player's input that has been transmitted.

Command: MODE_REFUSED
Payload:
    {
       reason: uint32
    }
Description:
    Inform a client that its request to change modes has been refused.

Command: CRC
Payload:
    {
       frame number: uint32
       hash: uint32
    }
Description:
    Informs the peer of the correct CRC hash for the specified frame. If the
    receiver's hash doesn't match, they should send a REQUEST_SAVESTATE
    command.

Command: REQUEST_SAVESTATE
Payload: None
Description:
    Requests that the peer send a savestate.

Command: LOAD_SAVESTATE
Payload:
    {
       frame number: uint32
       uncompressed size: uint32
       serialized save state: blob (variable size)
    }
Description:
    Cause the other side to load a savestate, notionally one which the sending
    side has also loaded. If both sides support zlib compression, the
    serialized state is zlib compressed. Otherwise it is uncompressed.

Command: PAUSE
Payload:
    {
       nickname: char[32]
    }
Description:
    Indicates that the core is paused. The receiving peer should also pause.
    The server should pass it on, using the known correct name rather than the
    provided name.

Command: RESUME
Payload: None
Description:
    Indicates that the core is no longer paused.

Command: STALL
Payload:
    {
       frames: uint32
    }
Description:
    Request that a client stall for the given number of frames.

Command: RESET
Payload:
    {
       frame number: uint32
    }
Description:
    Indicate that the core was reset at the beginning of the given frame.

Command: CHEATS
Unused

Command: CFG
Unused

Command: CFG_ACK
Unused

Command: PLAYER_CHAT
Payload:
    {
       nickname: char[32] (server-only)
       message: char[] (variable)
    }
Description:
    Sends a player chat message to the host and other connected clients.
    The server is responsible for relaying this chat message to all clients,
    including the one who sent it.

Command: PING_REQUEST
Payload: None
Description:
    Request the peer to answer back to this command.
    Used to calculate an estimated latency between peers.

Command: PING_RESPONSE
Payload: None
Description:
    Answer back PING_REQUEST commands.
    Used to calculate an estimated latency between peers.

Command: SETTING_ALLOW_PAUSING
Payload:
    {
       allow: uint32
    }
Description:
    Setting sharing command.
    Tell the client whether it is allowed to pause or not.

Command: SETTING_INPUT_LATENCY_FRAMES
Payload:
    {
       min: int32
       max: int32
    }
Description:
    Setting sharing command.
    Tell the client the input latency frames range it should be using.

Input types

Each input device uses a number of words fixed by the type of device. When
buttons are listed, they are listed from lowest bit to highest bit, i.e.,
reverse order. When insufficient buttons are present to represent
a full 32-bit word, the remainder are reserved and unused.

JOYPAD
    {
       B, Y, Select, Start, Up, Down, Left, Right, A, X, L, R, L2, R2, L3, R3
    }

MOUSE
    {
       unused, unused, Left, Right, Wheel up, Wheel down, Middle, Horizontal
       wheel up, Horizontal wheel down
       Y: int16
       X: int16
    }

KEYBOARD
   (5 words, see netplay_keys.h)

LIGHTGUN
   {
      unused, unused, Trigger, Cursor, Turbo, Pause, Start
      Y: int16
      X: int16
   }

ANALOG
    {
       buttons: uint32, as JOYPAD
       Left analog Y: int16
       Left analog X: int16
       Right analog Y: int16
       Right analog X: int16
    }