How

How does this work?

There are four components to this.  They each communicate slightly differently.  

  1. ServiceNow Instance
    1. Sends REST Request via ecc_queue
  2. Midserver
    1. Executes REST call to the AI In A Box Server
  3. AI In A Box Server
    1. Handles REST by hitting OpenAI Compliant API endpoint
    2. Handles Streaming Response from OpenAI Compliant API, and passes them along to ServiceNow 
  4. Large Language Model Server 
    1. Handles REST calls from AI In A Box Server