I’ve been doing some coding on AWS for a side project, and I ended up developing something that’s more generally useful so I’ve released it as open-source. Behold, SuperQueue!
SuperQueue is a thread-safe, SQS- and S3-backed queue structure for ruby that works just like a normal queue, except it’s essentially infinite because it uses SQS (and S3 optionally) on the back end.
To install, just “gem install super_queue”.
To create a new SuperQueue, pass it an options hash. Some options are required, and some are optional. When you’re done with it, you should ideally call “shutdown” on it to shut it down gracefully and preserve any data. Or, if you want to delete the SQS queue and any lingering data, call “destroy.”
I’m using this code in production, and it actually works. I’m actually about to increase my reliance on it by putting it in an even more sensitive role in my app, so we’ll see what happens.