A Quick and Dirty Tutorial for Writing Mastodon Bots in Ruby

My most recent project has been this Mastodon bot. Every n hours (where n == 8, for now) it toots an interesting section of the Mandelbrot Set.

Unfortunately, when getting to the actual bot part of the project, I found very little in the way of tutorials or examples in Ruby, so I'm writing down what I did while it's still fresh in my mind.

1. Create an Account

First, we need an account on a Mastodon instance. I'm using http://botsin.space, an instance for bot writers. This is pretty straightfoward: just fill in your details in the signup form and away you go.

2. Get Credentials

Getting credentials is an overly complicated process that (fortunately) there are tools to help with. Basically you:

  1. Register your application on the Mastodon instance. This gives you two big numbers (the client ID and client secret).

  2. Using these numbers plus the username and password of your bot's account, log in. This gives you your bot's access token. Hold on to this.

  3. From now on, the bot will use the access token to authenticate itself. You can invalidate the access token at any time from the bot account's settings (Settings -> Authorized Apps).

Unfortunately, the Ruby Mastodon client API library doesn't have an easy way to do this for you. After wrestling with it for a while, I ended up just using Darius Kazemi's registration app at https://tinysubversions.com/notes/mastodon-bot/. It's a nice little webapp that does everything reasonably securely. Just log into your instance and, while logged in, go to the above URL and follow the directions. (You do need to have curl installed and available, though.)

Failing that and if you have a Python installation handy, you can use the Python Mastodon module. Allison Parish's excellent guide for doing that is here.

3. Install the Mastodon API Gem

Assuming you have Ruby installed, it's a simple matter of typing

gem install mastodon-api

Unfortunately, as of this writing, there's a bug in the current release which has been fixed in development but which breaks the bot.

I worked around this by creating a Gemfile and listing a recent git commit as the version. (Did you know that bundle can install gems from GitHub? It can!)

It looks like this:

source 'https://rubygems.org'

gem "mastodon-api", require: "mastodon", github: "tootsuite/mastodon-api",
    branch: "master", ref: "a3ff60a"

Then, you cd to the project directory and run bundle:

bundle

(Or, if you want to install the gems locally and aren't using rvm or rbenv, you can install the gems locally:

bundle install --path ~/gems/

But you knew that, right?)

4. Basic Tooting

We are now ready to write a simple bot. This is done in two steps.

First, we create the client object:

client = Mastodon::REST::Client.new(
  base_url: 'https://botsin.space',
  bearer_token: 'bdc1fe7113d7cb9ea0f5d25')

The bearer_token argument is the token you got in step 2. (The real one will be longer than the one in the example.)

Then, we post the toot:

client.create_status("test status!!!!!")

And that should do it.

Here's the complete script:

require 'rubygems'
require 'bundler/setup'

require 'mastodon'

client = Mastodon::REST::Client.new(
  base_url: 'https://botsin.space',
  bearer_token: 'bdc1fe7113d7cb9ea0f5d25')

client.create_status("test status!!!!!")

One thing: because it's using bundler, the script must either be run from the directory containing the Gemfile or you will need to set the BUNDLE_GEMFILE environment variable to point to it.

I ended up doing the latter by writing a shell script to set things up and then launch the bot:

#!/bin/bash

botdir="$(cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"

export BUNDLE_GEMFILE="$botdir/Gemfile"
ruby "$botdir/basic_toot.rb" "$@"

The tricky expression in the second non-empty line extracts the directory containing the shell script itself; it figures out the location of the Gemfile from that.

5. Tooting With Media

Going from here to attaching a picture is pretty simple.

You set up the client as before and then upload the picture with upload_media():

form = HTTP::FormData::File.new("#{FILE}.png")
media = client.upload_media(form)

It's possible that you don't need to wrap your filename in a HTTP::FormData::File object (the API docs seem to imply that) but I wasn't able to get that to work and anyway, this doesn't hurt.

The media object you get back from the call to upload_media() contains various bits of information about it including an ID. This gets attached to the toot:

status = File.open("#{FILE}.txt") {|fh| fh.read}    # Get the toot text
sr = client.create_status(status, nil, [media.id])

It's in an array because (of course) you can attach multiple images to a toot.

Here's the entire script:

require 'rubygems'
require 'bundler/setup'

require 'mastodon'

FILE = "media_dir/toot"

client = Mastodon::REST::Client.new(
  base_url: 'https://botsin.space',
  bearer_token: 'bdc1fe7113d7cb9ea0f5d25')

form = HTTP::FormData::File.new("#{FILE}.png")
media = client.upload_media(form)

status = File.open("#{FILE}.txt") {|fh| fh.read}
sr = client.create_status(status, nil, [media.id])

This is mostly I use, plus the whole image plotting stuff, some logging stuff and a bunch of safety and error detection stuff.

6. Auth Token Management

You will note that I store the auth token as a string literal. I do this to simplify the example code but in general, it's a Bad Idea; mixing your credentials with your source code is just asking for your login details to get published on the open Internet. Keep them far away from each other.

For my bot, I use a YAML file that sits in the bot's work directory (which is not part of the source tree). Other options might be to pass it as a command-line argument or store it in an environment variable.

Other Resources

#   Posted 2017-12-15 03:46:21 UTC; last changed 2018-10-05 01:04:04 UTC


Site Updates

So I changed a couple of things on this website.

Firstly, I disabled Disqus comments from the blog. The service does a lot of cross-site shenanigans that have the potential to be a huge privacy violation. And anyway, nobody's ever used it except for me, the time I tested Disqus to see if it worked.

If you want to comment on a blog post, feel free to send me email and I'll update the post if I think it's relevant.

Secondly, I made a few small changes to the way some pages are laid out. This was part of completely rewriting how the non-blog parts of this site are rendered.

The website is generated from a collection of text files via a Perl script, all of which are stored in a git repository. Previously, that script invoked Infernal Icecube to do the rendering. The problem with this is that ii is pretty much abandoned, so updating the website usually involved first figuring out what dependency got left behind in the last system update. Also, ii wasn't in the repo, so I often had to first find a copy and install it. (It also doesn't help that macOS has a case-insensitive filesystem and the ii repo contains a file named ii and a directory named II in the root.)

I replaced it all with a script that renders using Text::Markdown, which I snagged and checked into the repo along with the rest of the site. Now, the only external dependencies are perl, make and rsync.

This meant that I had to manually convert my content from Infernal Icecube's markup format to Markdown. Fortunately, ii markup is basically an independent reinvention of Markdown, so that was pretty straightforward.

None of this should change how the site looks, but having had to review the entire site meant I saw its shortcomings anew, and so I've fixed a few things here and there. That's all.

#   Posted 2017-09-10 21:23:46 UTC; last changed 2017-09-10 21:25:14 UTC


RRSPs explained for Americans

Canada has a retirement savings vehicle called the Registered Retirement Savings Plan, (i.e. the RRSP). Money that is put into an RRSP is not taxed until you take it out, sometime after retirement. This isn't just pre-tax income: money that you put into an RRSP is actually not considered to exist for tax purposes.

So let's assume that Alice earns $100,000 a year and is taxed at a flat 10%1. Her employer withholds this money and sends it to Revenue Canada:

Earned          = $100,000
Tax             = $100,000/10 = $10,000

But during the year, Alice puts $10000 into an RRSP. When it comes time to compute her actual taxes for that year, the contribution is subtracted from her earned taxes, so as far as Revenue Canada is concerned, her tax status is:

Earned          = $90,000
Tax             = $90,000/10 = $9,000
Witheld         = $10,000
Refund          = $1,000

Because her employer has witheld too much taxes, she gets the rest back.

And this is where things get interesting. Alice, wanting to maximize her retirement savings, decides to also contribute the tax refund. She does this by borrowing2 $1,000 and contributing it before the end of the current year3. So now her taxes look like this:

Earned          = $89,000
Tax             = $89,000/10 = $8,900
Witheld         = $10,000
Refund          = $1,100

The extra $1,000 reduces her taxable income to $89,000 which adds another $100 to the refund. So she borrows another4 $100:

Earned          = $88,900
Tax             = $88,900/10 = $8890
Witheld         = $10,000
Refund          = $1,110

Which adds $10 to the refund. So she borrows another $10 to put in the RRSP:

Earned          = $88,990
Tax             = $88,990/10 = $8899
Witheld         = $10,000
Refund          = $1,111

At which point it's not really worth continuing. But the important thing is that she managed to get another $111 for free5.

And in real life where the tax rate is a lot higher than 10%, you'll also get correspondingly more bonus money, which is why this is a thing.

  1. The real tax rate is higher and more complicated; 10% keeps it simple without detracting from my main point. ↩

  2. The loan itself costs money, but it's pretty cheap. It lasts a couple of months and has the RRSP as collateral. ↩

  3. Actually, the deadline's the end of February of the following year. ↩

  4. More precisely, figures this all out in advance and adds it to the current loan. ↩

  5. Minus the cost of the loan, of course. ↩

#   Posted 2017-09-10 20:20:16 UTC; last changed 2017-09-10 21:44:18 UTC


Java Callbacks Ruby Style

Java (pre version 8, anyway) doesn't have proper anonymous functions1 for things like callbacks. Instead, the Java Way is to create an anonymous subclass implementing the behaviour you care about in an overridden method, then passing that instance to whatever needs it. So you see a lot of stuff like this:

addListener(new ChangeListener() {
    public void changed (InputEvent event, Actor actor) {
        do_a_thing(event, actor);
    }
});

ChangeListener is an abstract class in LibGDX implementing a bunch of callback hooks; this snippet passes addListener an anonymous subclass that overrides one of these callbacks and does what I want it to.

The Ruby Way, on the other hand, is to use blocks. This gives you the same power but requires less code. The equivalent (hypothetical) Ruby framework's interface might look something like:

addKeyDown {|event, keycode| do_a_thing(event, keycode)}

So this becomes jarring when you wish to call Java APIs from JRuby, as I did when writing a game in JRuby.

Initially, I just bit the bullet and wrote (non-anonymous) subclasses to hold the callbacks. Soon though, the number of callbacks got larger than my personal threshold (two) and I decided to try something else.

My first attempt was to create anonymous Ruby subclasses, which is fine but not really compact:

addListener(Class.new(ChangeListener) {
    def changed(event, actor)
      do_a_thing(event, actor)
    end
  }.new())

Another downside--and this is a bit more subtle--is that local variables defined in the method calling addListener are not visible to the new class. For example, this:

def listenForThing(someEvent)
    addListener(Class.new(ChangeListener) {
        def changed(event, actor)
          do_a_thing(event, actor) if someEvent == event
        end
      }.new())
end

won't work; changed can't see someEvent.

There had to be a better way--this is Ruby, after all! And there was.

Ruby provides the method define_method, which takes a block and name and turns it into a method which is then added to the class. I added a helper function to do the work:

def newListener(klass, name, &block)
  c = Class.new(klass)

  c.send(:define_method, name, &block)
  return c.new
end

and used it like this:

addListener(newListener(ChangeListener, :changed)  {|event, actor|
    do_a_thing(event, actor)
})

This is almost Ruby-like in it's compactness, but there's one subtle quirk.

The block has redefined self to be the owner of the new method (the sub-instance of ChangeListener created by newListener), so the method do_a_thing the block is trying to call should actually belong to that, not the class which called newListener in the first place.

This isn't what most people expect when they see a block in Ruby, and so could lead to all kinds of unpleasant surprises. It also optimizes for the less-common case and is kind of unwieldly to work around:

outerSelf = self
addListener(newListener(ChangeListener, :changed)  {|event, actor|
    outerSelf.do_a_thing(event, actor)
})

Fortunately, this is easy to fix. Instead of turning the block into the method body, I just define the new method as a simple one-liner that evaluates the block:

def newListener(klass, name, &block)
  c = Class.new(klass)

  c.send(:define_method, name) { | *args| block.call(*args) }
  return c.new
end

This way, the block keeps its old self value and everything works as expected:

addListener(newListener(ChangeListener, :changed)  {|event, actor|
    outerSelf.do_a_thing(event, actor)
})

And now, we have Java callbacks that are almost as easy as their Ruby equivalents.

  1. And you could argue that Java 8 doesn't have them either. ↩

#   Posted 2016-03-05 22:49:09 UTC; last changed 2016-03-13 18:57:21 UTC


Thoughts on game development with LibGDX and JRuby

As part of my ongoing, never-ending plan to Finally Write Another Roguelike, I've been dabbling with LibGDX, a game development library for Java. And having gotten to the point where it kind of does a few game-like things, I'm writing this to document my progress and in the hope that someone else may find this information useful.

LibGDX is a nice library but has the immediate major downside of requiring me to use Java without getting paid for it. I worked around that by using JRuby. (As you know Bob, JRuby is a version of Ruby that runs on the JVM and interoperates nicely with existing Java code.)

My first project was a class-by-class rewrite of Zombie Bird, a Flappy Bird clone written as a GDX tutorial. The code for that is here. (Feel free to grab it to use as a starting point.)

I then went on to port an existing Roguelike attempt in Ruby and ncurses to JRuby and GDX. This project is ongoing but I've managed to get it to do basic things (walk around, manage inventory) using GDX.

And during this process, I have managed to Learn Things. I will now impart my hard-won Knowledge to you.

Using Maven to Handle Java Dependencies

GDX has a GUI-based configuration thingy that will fetch the latest GDX jars and create an empty Gradle(?) project for you. Which is all well and good if you're using Java, but I'm not. Some Googling revealed that other people who've tried this just copied the jars somewhere into JRuby's classpath but I've gotten spoiled by the Maven repository network so I wanted something more automated.

JRuby has no problem accessing Java code in an external jar file, and GDX is accessible by Maven, so the solution was pretty straightforward:

  1. Create a trivial GDX program.
  2. Use Maven and the maven-assembly-plugin to build it into a big jar that includes all dependencies.
  3. Add the jar to my JRuby $CLASSPATH variable and require it.

(Why Maven and not one of the better other build tools? Because Maven is the one I know. You could do this with a better build tool or you could just clone my code from github and be done with it.)

My trivial GDX program was the default minimal program generated by one of the scaffolds (I don't remember which). One nice thing about it is that I can run it as a standalone program:

java -jar mvn_lib/target/bigjar-CURRENT-jar-with-dependencies.jar

It doesn't do much beyond display a moving image, but it's enough to prove to me that I successfully found all of the pieces GDX needs to run. This came in handy the times my program wouldn't work for some stupid reason because I could confirm that at least the library was correctly installed.

So now when I start on a fresh checkout, I just need to do a

cd mvn_lib; mvn clean package

and everything is there.

Upgrading GDX is simple too: just change the version in mvn_lib/pom.xml and rerun the build command.

Doing a GUI

GDX has a set of GUI widgets built on top of its 2D scene graph. As a GUI library, its, uh, pretty good for a game library.

Since I'm writing a turn-based game, having decent UI code is a lot more important to me than graphic performance and low-level control, and I started to regret using GDX for this project after a while. I persevered though and was eventually able to get a decent UI up and running. (Presumably, I'll need to do animations and sound effects at some point in the future, at which time I'll be thankful I stayed with GDX and didn't go to Swing or something.)

In any case, my game has a pretty simple screen layout: a map, a set of stats and a text window for game messages, arranged top to bottom:



(I'm using little pictures of characters because I'm oldschool. And a terrible artist.)

This was all pretty simple. The map is a subclass of com.badlogic.gdx.scenes.scene2d.ui.Widget and was pretty easy to code in Ruby. (Note: remember to super() in initialize, lest you get a really obscure crash.)

The others were just provided widgets (Table and TextArea respectively), each configured by a corresponding Ruby class. It would have been relatively simple to make them into subclasses, but I did them first and wasn't clear how well subclassing GDX components would work.

Getting the layout to work correctly was kind of painful and required a lot of trial-and-error intermixed with careful reading of the related Wiki pages. Turning on debug mode (by calling setDebug(true) on the table) helped a lot--this makes the table outline its cells with coloured lines and gives you a much better idea of how it works.

It also helped to be able to edit and run without needing to rebuild the project each time I made a small change, so that's one for Ruby1.

(This is not intended as a slam on the GDX UI code; this stuff is complicated. I have the same kinds of problems with Tcl/Tk and I've been using it on and off for decades.)

Don't Use Dialog

The one thing that ended up being a huge pain was the inventory window:



I initially wrote it as a subclass of com.badlogic.gdx.scenes.scene2d.ui.Dialog, which provides some nice functionality. Unfortunately, keystrokes sent to the inventory dialog would leak back to the main game. For example, pressing 'U' to stop equipping an item would then also cause the player to move up and to the right, which is what 'U' means during normal movement.

It turns out that pressing a key emits three events: KeyDown, KeyPressed and KeyUp, in that order. The Dialog was listening for KeyDown while the map listened for KeyPressed, so using a keyboard shortcut on the inventory window would close it on the KeyDown, returning control to the main game which would then receive the KeyPressed.

I wasn't able to find a way to make Widget deal with this correctly and ultimately ended up rewriting the inventory window as a subclass of Window, Dialog's parent.

Function Overloading

One major pain in using JRuby to call Java code is dealing with overloaded functions. The workaround is ugly, so it's fortunate that this comes up pretty rarely.

Here's the problem:

Suppose a Java class defines two functions with the same name and number of arguments but different types:

class Foo {
    int bar(int x) { ... }
    int bar(String x) { ... }
}

When calling them from Java, like this:

Foo x = new Foo();
...
x.bar(42);              // calls int version
x.bar("forty-two");     // calls string version

the Java compiler knows exactly which function to actually call because it knows the type of the argument. This is not the case in JRuby:

Foo.new.bar("thingy")   # may or may not work

Because Ruby is dynamically typed, JRuby can't tell which bar() to call, so it guesses and issues a warning.

Fortunately, JRuby provides the java_send method, which lets you call a method by name and type signature:

Foo.new.javaSend :bar, [java.lang.String], "thingy"

I've only needed to do this once in my project; most overloaded methods also have different numbers of arguments.

Implied Setters

Ruby (mostly) follows the Uniform Access Principle, while Java does not. As a result, the convention is for Java classes to implement getter and setter methods of the form setXXX() and getXXX().

However, JRuby automagically converts between the two. That is, this:

x.text = "some text"
puts x.text

works the same way as this:

x.setText("some text")
puts x.getText()

in a Java class that implements only setText() and getText().

I initially stuck to the Java form for Java classes because I didn't want to obscure the underlying implementation. However, the Ruby form is so much nicer that I eventually gave in. I'm not sure if that's a good idea or not.

Unsolved Stuff

Dynamic Constructors

Similar to method overloads above, JRuby has trouble with overloaded constructors. And like the method above, it provides a workaround:

construct = JavaClass.java_class.constructor(Java::int, Java::int, Java::int)
object = construct.new_instance(0xa4, 0x00, 0x00)

When I tried it, I did indeed get an object that claimed to be of the expected class, but I couldn't call the methods. It looked to me like it was an instance of the class but deep down, JRuby believed it was an instance of java.lang.Object.

It ended up that I didn't need that class at all, so I never investigated further, but I wasn't able to find out if this was a JRuby bug or something else I didn't understand.

Worst case, I can write a helper function in Java and include it in my part of the big JAR.

Building a Release

I haven't yet come up with a satisfactory way of creating a release suitable for end-users.

Warbler looks promising. I've played with it a little, but haven't yet gotten it to do what I want. Perhaps with more fiddling, I can get it to make a releasable build for me.

The other issue is that JRuby needs you to include the source code in the jar file. This won't be a problem if you're just going to release it as open-source software, but if you're trying to sell the game or even are just trying to prevent spoilers, you'll need to do something else.

One possible solution is to encrypt the Ruby sources and keep them as resources, then write a small Java loader that reads them in, decrypts them and hands them off to the JRuby API. Alternately, you could write a script to compile the Ruby sources into a Java array literal which, once again a loader routine decodes and hands off to the JRuby API. Either way, the result is a more-or-less ordinary Java program.

Overall, It's Not Bad

In general, I'm finding the JRuby+GDX combination to work pretty well. There's nothing insurmountable and the problem areas, where they exist, are pretty rare. Ruby itself is a much more powerful and expressive language than Java, and is a lot more fun to use.

  1. That being said, it takes about 6 seconds for JRuby to go from invocation to displaying that first window, so it's not lightning-fast either. There's a bunch of time that static typing and an IDE would have saved me too. ↩

#   Posted 2016-03-05 21:55:13 UTC; last changed 2016-03-05 22:30:58 UTC


Recent Posts:

Fallout 4: I am Disappoint

I'm Over There

Another blog post, another blogging program

Tweets of the Old Republic

Adventures in Python, Part 1:

Haloscan Gone

Sheesh!

Ten Best New (to me) Songs of 2009

Testing: software update

Another Project Announcement: nest