# Routing Assurance - Marvis CNI A/B Testing

Welcome to the Juniper Mist Routing Assurance Lab - Marvis CNI A/B testing.  

You will qualify for recognition and even gifts, so be smart, precise & focused.

## Overview

During the next 15 min or so, you´re going to help your customers, partners & yourself to get the best out of our next gen Natural Language Assistant for Juniper Public Documentation.  

As simple as 1-2-3:

-   Ask questions
-   Read answers & references for the competing LLM
-   Score them! 

## Starting Lab

This lab uses Mist and therefore will require an invite to the Organization in Mist. If you ran the Routing Assurance, you should have an instance to do A/B testing.

If not, please send an email to JMRA-demo@juniper.net , requesting the access to Routing Assurance Demo Org.  

## Come up with Questions

To help us train the model, we need you to come up with a minimum of 15 questions or more. Think from the perspective of a Juniper SE, Partner, Prospect, and Customer. We would the questions to be related to Routing, AWAN Portfolio, and Routing Assurance.

## Ask Questions

1.  On the left panel go to **Organisations > InventorRAG Evaluation**

    ![RAG_Eval](./images/RAG_Eval.png)

2.  Type a question in the "Prompt" Box and click **Go** button in the upper right. After a few seconds answers are displayed for LLM solution A and B:

    ![llm_response](./images/llm_response.jpg)

3.  Rate each answer using Thumbs Up/Down with the following in mind:

    -   Quality/accuracy/relevance of the summary
    -   Incomplete answers should be rated Thumbs Down
    
    
    ![rate_response](./images/rate_response.jpg)

4.  Include a text comment to describe the issues, specially when rating thumbs down, and Click on **Submit Feedback**.

    ![submit_feedback](./images/submit_feedback.png)

5.  Repeat these steps for each of your 15 or more questions.

## Conclusion

In our thrilling A/B testing competition, three winners will be chosen to receive exclusive prizes that recognize their exceptional contributions. Are you ready to be one of the pioneering minds leading the charge into a new era of AI excellence?

![prizes](./images/prizes.jpg)

-   1st Prize - Explore the Universe - 5,000 Points
-   2nd Prize - Rocket to Space - 2,500 Points
-   3rd Prize - Fly to the Moon - 1,500 Points

Thanks a lot for your time & efforts, we value your contribution to make our Conversational Interface better for Juniper. 
 
Please send us feedback about your experience at JMRA-demo@juniper.net