# 5 minute challenge: compare different LLMs

Let's compare how different large language models are doing when encounter the same task

1. Go to <https://app.scade.pro/flow/> and click the **New flow** button and then choose **Start with blank**

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FzVuqpcuwgY0Z9MiAUeTe%2FScreenshot%202024-03-27%20at%2007.07.21.png?alt=media&#x26;token=472a1d2a-91eb-45da-b476-2d5de2e8a3d0" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FvmfhsMwpn9FBg2bSsTo4%2FScreenshot%202024-03-27%20at%2007.07.35.png?alt=media&#x26;token=e8696bc1-ebad-4304-981a-2d003a91288e" alt=""><figcaption></figcaption></figure>

2. There will already be **Start** end **End** nodes on the canvas. In this particular case we don't need further integrations so we can simply delete the **End** node, but it's not a general rule.\
   \
   Click on the **End** node to highlight it and hit backspace

<div><figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FJPnjTrdF3dpC9KgXEARX%2FScreenshot%202024-04-03%20at%2010.24.55.png?alt=media&#x26;token=9c8a2b88-42f1-4d42-b30b-b07a5f09cd9f" alt=""><figcaption></figcaption></figure> <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2F9AqctaxDitI98xss35GA%2FScreenshot%202024-04-03%20at%2010.25.10.png?alt=media&#x26;token=c7b0cef8-bdcc-4550-86d9-af78fb141394" alt=""><figcaption></figcaption></figure></div>

3. Go to the **Settings** of the **Start Node Form** and click **Configure fields** to start adding your data

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2F4NPTOFhX1XfBHXpTipaV%2FScreenshot%202024-04-03%20at%2010.27.31.png?alt=media&#x26;token=641c44a1-81d3-409f-a60b-fc3e9471595e" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FX0iY90eH3JHS5eGKslvM%2FScreenshot%202024-04-03%20at%2010.28.44.png?alt=media&#x26;token=bf998aa8-4eec-4753-b331-9e03f8280527" alt=""><figcaption></figcaption></figure>

4. Add one field, call it, for example, `prompt`. Click **Save**

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FUL1Spcfab88aceT5cGLd%2FScreenshot%202024-04-03%20at%2010.29.33.png?alt=media&#x26;token=d71621a0-5da7-4696-827f-6de2f3b542c0" alt=""><figcaption></figcaption></figure>

5. Add your prompt to the **Start** node. For example, let's ask a question that would be easy for a human but can be tricky for AI: `how many N letters are there in some words, like "banana" or "sky"?`\
   \
   Hit the **Execute** button and then click **Save and execute** to run this node and generate the output. We need it to move further.

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2F3zYbJOXxbJ0Xn1YxQqaJ%2FScreenshot%202024-04-03%20at%2010.41.40.png?alt=media&#x26;token=9150814f-f248-41a0-8f11-3489c7a83ae8" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FiITMuErdmO8eYIrXOnJx%2FScreenshot%202024-04-03%20at%2010.41.52.png?alt=media&#x26;token=f687fdf2-8c03-4ba7-9b3a-e17b0b2709e2" alt=""><figcaption></figcaption></figure>

6. Use the search panel on the left to find processors that we will compare. Let's try **ChatGPT 3.5**, **ChatGPT 4**, **Claude 2** and **Claude 3**. \
   \
   Simply type name of the model (`ChatGPT` and `Claude`) in the panel and drag the processors to your workspace. We need two instances of the **ChatGPT** and two instances of the **Anthropic/Claude**

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FhchhEIoAl5s6ZjaMZQrW%2FScreenshot%202024-04-03%20at%2010.55.33.png?alt=media&#x26;token=45c1d721-a079-4743-a99b-f06c34ff4905" alt=""><figcaption></figcaption></figure>

7. Once we have all four nodes of our LLMs, let's set models versions and name nodes accordingly\
   \
   Pick one of the **ChatGPT** nodes and go to the **Settings**. Locate the **Model** section and choose `gpt-4`  <br>

   <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FClTjfhTuH8jHXH8uL4L5%2FScreenshot%202024-04-03%20at%2011.07.34.png?alt=media&#x26;token=991fb008-68d7-4c78-8800-f2a11b745008" alt=""><figcaption></figcaption></figure>

   \
   \
   To rename the node hover over its title and click the pencil<br>

   <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2Fo5tdLFMGUALNFWxpjPdz%2FScreenshot%202024-04-03%20at%2012.15.42.png?alt=media&#x26;token=b3e943d0-c30f-4454-9c8a-19f37e5466e7" alt=""><figcaption></figcaption></figure>

   \
   \
   Let's name this node **ChatGPT 4** for clarity

   <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FXz2zQEqJWH7K2cdgyKQP%2FScreenshot%202024-04-03%20at%2012.22.01.png?alt=media&#x26;token=c42aa916-38da-44b6-a015-11ea8f6acbbe" alt=""><figcaption></figcaption></figure>

8. We don't have to change the model version of the next **ChatGPT** node, it has 3.5 set by default. Let's simply rename it to **ChatGPT 3.5**

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FwmM0nFHXs9pRSUQLfWof%2FScreenshot%202024-04-03%20at%2012.23.49.png?alt=media&#x26;token=82dd105e-ec4a-4551-8d5e-605b853f126b" alt=""><figcaption></figcaption></figure>

9. Following suit, let's change the model version (for example, to `claude-3-opus`) of one of the **Anthropic/Claude** nodes and rename both of them to make it obvious

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2Fij3wVx1b1xSoFCCZnZ1y%2FScreenshot%202024-04-03%20at%2012.30.42.png?alt=media&#x26;token=c88338da-768e-4a18-ab61-6063261930e5" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FLeN9ZNJuTpiB8JPYzEOO%2FScreenshot%202024-04-03%20at%2012.33.44.png?alt=media&#x26;token=dd6e88ca-f09b-4de0-854b-51cd1d4e1f50" alt=""><figcaption></figcaption></figure>

10. Connect the output of your **Start** node with the inputs of every other node

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FmWwQnIPPASeWgra3jeYa%2FScreenshot%202024-04-03%20at%2012.37.51.png?alt=media&#x26;token=a32000c3-231d-4a10-aab9-65c01d1e7f59" alt=""><figcaption></figcaption></figure>

11. We are not done here yet. We need to go to each **ChatGPT** and **Claude** node and add messages. Go to the settings of the node, locate the **Messages** section, click the pencil and hit **Add message** after that\ <br>

    <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FeJusan8rzr7D0V5JLIL2%2FScreenshot%202024-04-03%20at%2012.47.46.png?alt=media&#x26;token=c5907c99-72e2-4adc-8f9d-1785ed1e2d89" alt=""><figcaption></figcaption></figure>

    <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FyuGJeGvTB4Cc72qAbelH%2FScreenshot%202024-04-03%20at%2012.49.30.png?alt=media&#x26;token=bdebc78e-e36a-4cb0-873c-2434f544e4d6" alt=""><figcaption></figcaption></figure>

    \
    \
    If we work with LLM nodes, we need to use the **Expression editor** for obtaining data from other nodes. After adding the message, click the **#** symbol, and the **Expression editor** will open<br>

    <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FyZI0MkNVBmyaH12Z4RkR%2FScreenshot%202024-04-03%20at%2012.53.13.png?alt=media&#x26;token=c17a65ef-3204-4e68-ad8b-e217e090d5ff" alt=""><figcaption></figcaption></figure>

    \
    \
    Locate the **Start Node Form** on the left, click on it and drag the `prompt` to the expression field<br>

    <figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2F8PygiWgNzs4HU8zmLgzj%2FScreenshot%202024-04-03%20at%2012.56.05.png?alt=media&#x26;token=52c27e98-e936-48ad-a94c-9a0f46931033" alt=""><figcaption></figcaption></figure>

    \
    \
    Don't forget to click Save everywhere and repeat this course of action for all other **ChatGPT** and **Claude** nodes<br>
12. All set! Hit the Play button on the top panel

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FrFwo1Fx2Ioz0ij8oRf7t%2FScreenshot%202024-04-03%20at%2013.03.26.png?alt=media&#x26;token=b4ad79cc-2926-42f3-9c54-9fa20184ea7e" alt=""><figcaption></figcaption></figure>

13. Funny enough, **ChatGPT 3.5** done better than **ChatGPT 4** in this task. **Claude 3** is also completed the task

<figure><img src="https://2793209830-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNCrOWKpBlXYnNB9SFIzb%2Fuploads%2FTCbY30ZxU7eYoynqAymS%2FScreenshot%202024-04-03%20at%2013.06.39.png?alt=media&#x26;token=52b32ef8-da21-4560-8ab9-866d3174d709" alt=""><figcaption></figcaption></figure>

You can test any models in similar way and pick one that suits you best
