baucord Making a bot

Making a bot

A bot is a program that joins your server as a member of its own: it reads the chat, answers, and plays audio in a voice channel. There is a music bot you can run as it is, and a door for writing your own in any language — thirty lines of Python is a bot.

What a bot is

To the server a bot is a person with a device: it has its own key, its own name in the roster, and the role its invite code gave it. It can do what that role allows and nothing else — a guest can join channels, speak and write in General, a member can also answer direct messages. There is no bot flag yet (what is coming), so it shows up like anyone else, and you kick, ban or mute it like anyone else.

Two shapes, pick by what you want to do:

What you need

The music bot

The bot is the program music built above. It needs the server's address, a name, a file to keep its key in, and an invite code the first time: 

./target/release/examples/music --server 203.0.113.5:51000 --name MusicBot \
    --key musicbot.key --invite K7FQ-2MX9-ABCD --library ~/Music
Flag (environment variable) What
--server (BAUCORD_SERVER) The server's address and port, as people type it in the app.
--name (BAUCORD_BOT_NAME, default MusicBot) The bot's name in the roster.
--key (BAUCORD_KEY, default musicbot.key; the gateway's is bot.key) Its identity, created on the first run. Whoever has this file is the bot: keep it like a password, and keep it — a new key is a new member needing a new invite.
--invite (BAUCORD_INVITE) An invite code. Needed on the first run only: it links the key, and from then on the key alone logs in. Leaving the flag in place afterwards is harmless.
--pin (BAUCORD_PIN) The server's identity (SHA256:…, printed by the server at start; Settings → Servers in the app). Without it the first connection's identity is trusted and written to a .pin file beside the key (musicbot.key → musicbot.pin), and every later run insists on it — what the app does.
--library (BAUCORD_LIBRARY) The folder !play <file> may read from, and never outside it. Without the flag, links only.
--voice-kbps (BAUCORD_VOICE_KBPS, default 64) Opus bitrate of what it plays, 16–128.

Its log says who it is, which identity it pinned, when it was admitted, and what it is playing. Then, in the server's chat:

In General What happens
!play song.mp3 Joins the voice channel you are in and plays the file from the library (a path under it: !play albums/x/03.flac). You have to be in a voice channel; it says so otherwise.
!play https://…/track.mp3 Plays a direct link to an audio file or stream: an mp3, ogg, flac, m4a or wav somewhere on the web, or an internet radio stream, which plays until !stop. It is fetched as it plays, so there is no waiting for a download.
!play … while something plays Queues it; the bot answers with its place in the line.
!skip, !stop, !queue, !help Next track; stop and drop the queue; how many are waiting; the list of commands.

A YouTube, Spotify, SoundCloud or Bandcamp link is the page about a track, not the track; the bot says so and plays nothing. Those sites need an extractor that changes monthly, which is yt-dlp's job — see YouTube and the sites yt-dlp knows.

It says little in chat on purpose: the server's chat limit (five lines in five seconds) applies to a bot as to anyone, and a chatty bot loses the lines that matter.

A bot of your own, in Python

Two programs: the gateway, which is the member of the server, and your bot, which talks to the gateway. Start the gateway with the same flags as the music bot (no --library): 

./target/release/baucord-gateway --server 203.0.113.5:51000 --name HelloBot \
    --key hellobot.key --invite K7FQ-2MX9-ABCD

It listens for your bot on 127.0.0.1:8765 (--listen to change it; see keeping it running before you listen on anything but the loopback). Then the bot — this one answers !ping and greets whoever joins. pip install websockets, save it as echo_bot.py, run it: 

import json
from websockets.sync.client import connect

with connect("ws://127.0.0.1:8765/") as ws:
    hello = json.loads(ws.recv())
    print("gateway", hello["d"]["gateway"], "connected", hello["d"]["connected"])
    while True:
        f = json.loads(ws.recv())
        if f["op"] != "dispatch":
            continue
        if f["t"] == "MESSAGE_CREATE" and not f["d"]["own"] and f["d"]["text"] == "!ping":
            ws.send(json.dumps({"op": "action", "t": "CREATE_MESSAGE", "d": {"text": "pong"}}))
        if f["t"] == "MEMBER_ADD":
            ws.send(json.dumps({"op": "action", "t": "CREATE_MESSAGE",
                                "d": {"text": "hi " + f["d"]["name"]}}))

That is the whole shape of a bot: read frames, act on the ones you care about, send actions. Type !ping in the server's chat and the bot answers pong. The bot's own lines come back to it too, marked own — that check is what keeps it from answering itself.

The gateway can be left running while you restart the bot as often as you like; the bot's connection to it is the only thing that drops. Several bots may connect to one gateway; they share its one identity and all see every event. One bot per gateway is the normal shape.

What a bot sees and does

One WebSocket; text frames carry JSON, binary frames carry audio. Every JSON frame is {"op": …, "t": …, "d": {…}}: op is the kind of frame, t its name, d its fields. Numbers that are 64-bit (client_id) travel as strings of digits and user ids as hex strings, so JavaScript keeps every bit; message ids, timestamps (milliseconds) and channel ids are plain numbers.

From the gateway

The first frame is {"op":"hello"}, with the gateway's version, whether it is connected, and the READY fields (below) if it is. Then, as its session comes and goes, {"op":"status"} frames: CONNECTED, DISCONNECTED (with reconnecting), REFUSED (a bad code, a ban), REMOVED (a kick). A frame it could not use comes back as {"op":"error","d":{"message":…}}: unknown JSON, a missing field, an action sent before the session is ready, or a refusal the server answered with words.

Events: {"op":"dispatch","t":…}

t Fields in d When
READY client_id, user_id, permissions, server_name, greeting, channels (id, name, has_password, user_limit), members (client_id, name, channel_id, user_id, roles, level) Admitted, roster received. channel_id is null in the lobby.
MESSAGE_CREATE id, ts, client_id, user_id, name, text, conv ("general" or {"dm": user_id}), own, attachment when it is a file A line in General, a direct message to the bot, a file.
MESSAGE_UPDATE, MESSAGE_DELETE id, text / id An edit, a deletion.
MEMBER_ADD, MEMBER_REMOVE, MEMBER_UPDATE client_id, name, channel_id; what changed Someone connected, left, was renamed or given a role.
VOICE_STATE client_id, channel_id Someone moved channel — the bot too, after JOIN_CHANNEL.
CHANNELS, JOIN_DENIED channels, memberships / channel_id, reason The channel list changed; a join the server refused.
SPEAKING, AUDIO_STATE client_id, speaking / muted, deafened The lights in the roster.
RATE_LIMITED retry_ms The bot wrote too fast; that line was dropped.
HISTORY conv, messages, more The answer to FETCH_HISTORY.
VOICE_REPORT received, expected, lost, late, rtt_ms Every 2 s while the bot plays: what the server got of it.
RESTRICTED, ROLES, MOVED_IDLE, SHARE_START, SHARE_STOP … A mute on the bot; the role table; moved to the AFK channel; screen shares (it cannot watch them).

Actions: {"op":"action","t":…}

Each becomes the message a person's app would send, and the server checks the bot's permissions as it would anyone's.

t Fields in d
CREATE_MESSAGE text — a line in General.
CREATE_DM user_id, text, client_ref (optional; comes back in MESSAGE_ACK or MESSAGE_FAILED).
EDIT_MESSAGE, DELETE_MESSAGE id, text / id — its own messages.
JOIN_CHANNEL channel_id, password (optional). Audio goes to the channel the bot is in: join one before playing; the lobby's audio is dropped.
SET_NICKNAME, AUDIO_STATE, SPEAKING name; muted, deafened; speaking (the light; playing sets it by itself).
FETCH_HISTORY, MARK_READ conv, before, limit (50, at most 200); conv, id.

The complete list, with every field, is in the gateway's README. The names follow Discord's where the thing is the same, so a bot written for Discord ports without relearning them.

Playing audio

A binary frame is sound: 32-bit little-endian floats, 48 kHz, mono, between −1 and 1, any length. The gateway cuts it into 20 ms pieces, encodes them as Opus and sends one every 20 ms, so the bot may push a whole song at once: up to ten seconds ahead, after which its socket waits. That back-pressure is how a decoder's output paces itself to the room — ffmpeg … | bot works out on its own. The speaking light goes on with the first frame and off 200 ms after the last. {"op":"voice","t":"STOP"} drops what is queued.

From anything ffmpeg can read, this is the whole conversion; stereo is the bot's to mix down (-ac 1), the apps play mono: 

ffmpeg -loglevel error -i song.flac -f f32le -ar 48000 -ac 1 -

Join the asker's channel first: their channel_id is in READY's roster, and VOICE_STATE events keep it current. A bot hears nothing yet (what is coming).

YouTube and the sites yt-dlp knows

music_bot.py, next to the echo bot in the examples, is the music bot for links the Rust one refuses: !play <url> runs yt-dlp, pipes it through ffmpeg into the gateway, and joins the asker's channel; !stop stops. It needs pip install websockets, yt-dlp and ffmpeg on the path, and a gateway running as above. Its ninety lines are a fair template for any bot that plays something.

Why two music bots: an extractor for those sites changes monthly and is a project of its own, which yt-dlp is. The Rust bot stays small and needs nothing installed; the Python one knows every site yt-dlp does, at the price of two more programs to keep updated.

In Rust, as one program

The gateway is also a library, and the music bot is built on it: no second process, the bot is the member. With the gateway crate as a dependency: 

use gateway::{bot::Bot, net::Config, wire::{Action, Outbound}};

let bot = Bot::connect(config, 64)?;          // threads of its own; returns at once
for event in bot.events() {                   // Outbound::Status / Dispatch / Error
    if let Outbound::Dispatch { t: "MESSAGE_CREATE", d } = &event
        && d["text"] == "!ping"
    {
        bot.act(Action::CreateMessage { text: "pong".into() });
    }
}

bot.play(&samples) plays 48 kHz mono floats in the bot's channel and blocks while ten seconds are queued, so a decoder calling it runs at the room's pace; bot.stop_playing() cuts it within a frame; bot.shutdown() says goodbye to the server. Admission, the pin and reconnecting are the gateway's, shared. The music bot's source, crates/gateway/examples/music.rs, is the worked example: decoding with symphonia, resampling with rubato, a queue, and the chat thread.

Keeping it running

Rules and limits

What is coming

In this order: a bot role and /bot create, so a bot is marked as one and its code is minted from the app; slash commands with replies only the asker sees; audio in (what the room says, for a recorder or a soundboard that reacts); a way for a bot to opt out of the roster's lights; and the gateway and the music bot in the downloads and the Docker image, so nothing has to be built.