Routing Assurance - Marvis CNI A/B Testing

Welcome to the Juniper Mist Routing Assurance Lab - Marvis CNI A/B testing.

You will qualify for recognition and even gifts, so be smart, precise & focused.

Overview

During the next 15 min or so, you´re going to help your customers, partners & yourself to get the best out of our next gen Natural Language Assistant for Juniper Public Documentation.

As simple as 1-2-3:

  • Ask questions

  • Read answers & references for the competing LLM

  • Score them!

Starting Lab

This lab uses Mist and therefore will require an invite to the Organization in Mist. If you ran the Routing Assurance, you should have an instance to do A/B testing.

If not, please send an email to JMRA-demo@juniper.net , requesting the access to Routing Assurance Demo Org.

Come up with Questions

To help us train the model, we need you to come up with a minimum of 15 questions or more. Think from the perspective of a Juniper SE, Partner, Prospect, and Customer. We would the questions to be related to Routing, AWAN Portfolio, and Routing Assurance.

Ask Questions

  1. On the left panel go to Organisations > InventorRAG Evaluation

    RAG_Eval

  2. Type a question in the “Prompt” Box and click Go button in the upper right. After a few seconds answers are displayed for LLM solution A and B:

    llm_response

  3. Rate each answer using Thumbs Up/Down with the following in mind:

    • Quality/accuracy/relevance of the summary

    • Incomplete answers should be rated Thumbs Down

    rate_response

  4. Include a text comment to describe the issues, specially when rating thumbs down, and Click on Submit Feedback.

    submit_feedback

  5. Repeat these steps for each of your 15 or more questions.

Conclusion

In our thrilling A/B testing competition, three winners will be chosen to receive exclusive prizes that recognize their exceptional contributions. Are you ready to be one of the pioneering minds leading the charge into a new era of AI excellence?

prizes

  • 1st Prize - Explore the Universe - 5,000 Points

  • 2nd Prize - Rocket to Space - 2,500 Points

  • 3rd Prize - Fly to the Moon - 1,500 Points

Thanks a lot for your time & efforts, we value your contribution to make our Conversational Interface better for Juniper.

Please send us feedback about your experience at JMRA-demo@juniper.net