5 minute challenge: compare different LLMs

Let's compare how different large language models are doing when encounter the same task

1. Go to https://app.scade.pro/flow/ and click the New flow button and then choose Start with blank

2. There will already be Start end End nodes on the canvas. In this particular case we don't need further integrations so we can simply delete the End node, but it's not a general rule. Click on the End node to highlight it and hit backspace

3. Go to the Settings of the Start Node Form and click Configure fields to start adding your data

4. Add one field, call it, for example, prompt. Click Save

5. Add your prompt to the Start node. For example, let's ask a question that would be easy for a human but can be tricky for AI: how many N letters are there in some words, like "banana" or "sky"? Hit the Execute button and then click Save and execute to run this node and generate the output. We need it to move further.

6. Use the search panel on the left to find processors that we will compare. Let's try ChatGPT 3.5, ChatGPT 4, Claude 2 and Claude 3. Simply type name of the model (ChatGPT and Claude) in the panel and drag the processors to your workspace. We need two instances of the ChatGPT and two instances of the Anthropic/Claude

7. Once we have all four nodes of our LLMs, let's set models versions and name nodes accordingly Pick one of the ChatGPT nodes and go to the Settings. Locate the Model section and choose gpt-4

   

   To rename the node hover over its title and click the pencil

   

   Let's name this node ChatGPT 4 for clarity

   

8. We don't have to change the model version of the next ChatGPT node, it has 3.5 set by default. Let's simply rename it to ChatGPT 3.5

9. Following suit, let's change the model version (for example, to claude-3-opus) of one of the Anthropic/Claude nodes and rename both of them to make it obvious

10. Connect the output of your Start node with the inputs of every other node

11. We are not done here yet. We need to go to each ChatGPT and Claude node and add messages. Go to the settings of the node, locate the Messages section, click the pencil and hit Add message after that

    

    

    If we work with LLM nodes, we need to use the Expression editor for obtaining data from other nodes. After adding the message, click the # symbol, and the Expression editor will open

    

    Locate the Start Node Form on the left, click on it and drag the prompt to the expression field

    

    Don't forget to click Save everywhere and repeat this course of action for all other ChatGPT and Claude nodes

12. All set! Hit the Play button on the top panel

13. Funny enough, ChatGPT 3.5 done better than ChatGPT 4 in this task. Claude 3 is also completed the task

You can test any models in similar way and pick one that suits you best