My Chess plugin has now been on the ChatGPT plugin store for almost 2 weeks. So I thought it might be fun to run through some stats and initial impressions.

My plugin was approved on the 11th of May.

Just in them for them to start rolling out access to plugins to all ChatGPT+ users!

I wasn’t expecting a huge number of users - chat isn’t the best way to play chess, though it does have some advantages with ChatGPT being able to explain what’s happening in the game.

What I was expecting was a few users trying it out. Maybe a few games from people trying out the plugin store. At the time of writing, there are 10,167 games in the database.

Here are the daily stats for new games started:

Initially, things were pretty crazy, peaking at around 1,800 games in one day.

Now, you might be wondering what happened between the 19th and the 22nd. It looks like everyone stopped using the plugin.

There’s a slight issue/feature with the plugin store at the moment. When you change metadata about your plugin it will automatically switch to an “unverified” state while the new metadata is reviewed - this can take up to 7 days. Surprisingly for an AI company, this is a manual process.

This has the unfortunate effect of removing your plugin from the store - and for anyone who has it installed, you get a lovely red shield next to the plugin.

This is quite unnerving for users and a lot of people have been uninstalling plugins that switch to unverified. Hopefully, as the plugin store matures these issues will be resolved.

At the time my plugin was out of the store a lot more plugins rolled out and I think some of the excitement around plugins tailed off.

I’m expecting the number of games to continue to drop off as more plugins are released and the novelty wears off.

What colour do most people like to play - everyone likes to play as white - there is a small advantage to playing white in chess - is everyone afraid of the AI?

How many moves are there in a typical game? As mentioned earlier - a chat interface may not be the most appealing way to play chess, so I would expect the number of moves to be low.

I was pleasantly surprised to see that people are playing quite a few moves per game. On average about 10 moves per game.

Did anyone beat the computer? Did anyone even try?

So, it looks like people are doing pretty well - what level are they playing at?

Well, it looks like most people are playing at the lowest level!

How much is it costing me to run?

I’m using the serverless framework and running everything on AWS using Lambdas with CloudFront for caching, all the data is stored in DynamoDB.

The most expensive service so far has been DynamoDB - this has cost me an unbelievable $0.14 so far. My prediction is this will average about twice this per month.

There are probably many ways to optimise this, but I’m not too worried about this cost at the moment.

I’ve not managed to get CloudFront costs over 1 cent yet. The free tier is very generous with 1TB of data transfer and 2 million requests per month.

For the Lambda functions we’re also managing to stay within the free tier. Though we are getting close to breaking out of that. We’ll see once we have a full month how much this costs - but I’m expecting it to be less than $1.

The most expensive part of the system is storing secrets on AWS Secrets Manager - this is costing me $0.80 per month to store two secrets.

My impressions

There are still some rough edges on the plugin store - there’s currently no search or categorisation, so discoverability is very poor. There are improvements in the works around this. But as with the existing App Stores - it’s down to the plugin developers to do their own marketing.

Getting your plugin into the “popular” section will boost your number of users. It’s very much like the mobile App Stores - getting noticed by the guys running the shop is like winning the lottery. But - there’s no monetisation yet - so maybe it’s best not to get featured just yet…

If you do fancy a game of chess, you’ll find my plugin languishing on page 18 of the plugin store!