For most people ChatGPT is the go-to AI of choice for generative text, you give it a prompt and it spits out a result. Sometimes these results can be less than satisfactory leading to either improving the prompt input to get a different result or looking for alternative AIs.
Sometimes that alternative search can send you down a rabbit hole that can take hours and hours of research leading you to try different platforms, and different models, sign up for premium services or rewind back to the original platform you were on to start with. AI is moving so fast that every week there seems to be some new platform fighting for the top position or some new language model that is the bee's knees to be using but comparing them can be a full-time job if you just want to figure out what is worth looking at and what isn’t.
Enter Nat Friedman, former CEO of GitHub & Founder of AI Grant. Nat has recently released an extremely useful tool that is available at nat.dev that allows you to compare over 25 different AI models in an open playground environment.
The best thing about this web app is just how clean and simple it is to use. Simply put in the prompt, select the models, and the outputs are generated live for you. No doubt over time the list of models will grow but this is a spectacular resource to use if you are new to AI and on the hunt for something outside of ChatGPT.
As you can see in the above screenshot we have 4 outputs from the same prompt. The answers are all very different and it is worth finding a model that you like the answers from because some a clearly better than others. Another consideration is the operating cost and the speed of the AI models which are going to have a large impact on whether you adopt or integrate it into a workflow, this website shows you the speeds and character count.
It does allow you to fine-tune a lot of the model parameters should you want to dive a little deeper.
To be honest I hadn't heard of half of the models that were on the list but when I started playing with them it was no wonder because some of them were clearly not ready for mainstream adoption. All of the models knew when the Titanic sank but half of them really struggled with some of the slightly more curly questions that I threw at it.
I think this is a great tool to cross-check & reference data in the early stages of AI projects. Some AI models have a bad habit of confidently stating false facts that sounds plausible but are outright misinformation, nat.dev can help you quickly figure out which models are the most accurate. If you want to create a truthful AI that still sounds good you could mix the models and write content with the one you find is most natural and cross-reference the facts of the others. As you can see in the above photo a simple cross-check with a different tool will quickly improve the validity of outputs.
I can’t wait to see this website evolve over time as it is already on my favorites list. Over time I think this resource is only going to get better as they add more models into the mix like LLaMa & more advanced credit calculation options to allow people to forecast running costs as some people will want to run their own servers. Hopefully, they will expand it to allow even more advanced AI features like multi-modal inputs & outputs so we can get more than just text, no doubt people will want to compare GPT-4 the moment it goes live.
