GreenKey

Scribe Real-Time Dictation Server (SQCServer)

Version 3.2 Documentation

An overview and best practices for using GreenKey Scribe to capture audio, interpret quotes and trades, use skills built in the Scribe Discovery Engine, or deploy general real-time transcription.




Deploying SQCServer

The Scribe Dictation server (SQCServer) can be launched on any Docker-compatible machine with at least 5 CPUs and 5 GB RAM for each 5 connection instance of SQCserver.

All of Scribe's services require an authorized account. Contact us if you are interested in obtaining an account.




Step 1: Install Docker

Follow the instructions here to install Docker on your machine.




Step 2: Download the SQCServer Docker image

$ docker login -u [repository-user] -p [repository-password] docker.greenkeytech.com
Login Succeeded
$ docker pull docker.greenkeytech.com/sqcserver
$ docker pull docker.greenkeytech.com/scribelicense

The credentials provided by GreenKey should include your repository-user and repository-password.




Step 3: Launch the Scribe License Server

The Scribe License Server must have access to the internet and must have access to the shared folder above. Launch it using the command below:

docker run -d \
    -v "$PWD/[license-dir]":/gktlicense \
    -v "$PWD/[gkt-folder]":/gkt \
    -e GKT_USERNAME=username \
    -e GKT_SECRETKEY=secretkey \
    -e GKT_API="https://scribeapi.greenkeytech.com/" \
    docker.greenkeytech.com/scribelicense

where:

[license-dir] is the path to the gktlicense folder.*

[gkt-folder] is the path to the gkt folder.*

username is your GreenKey Scribe Username

secretkey is your GreenKey Scribe SecretKey

* The gktlicense folder and the gkt folder are directories that will be shared by the Scribe License container and by the SQC container. If you do not have these folders already, Docker will create them when you execute the run script. If you want these directories to be put in a custom location, make sure to specify the absolute path to the directory, and use quotes around the file path if there are any folder names with spaces in them.

NOTE: Only one license server can be created per credential pair. If another license server is started, the first one will fail to authenticate afterwards.

If you have an on-premise deployment, you can run the license server on an instance with internet access, then rsync the /gkt and /gktlicense folders with the isolated deployment. The folders must be synced once every 24 hours.

If you need to proxy the connection to our API, specify the following environmental variables:

  -e https_proxy="http://some.proxy:some.port" \
  -e http_proxy="http://some.proxy:some.port" \




Step 4: Launch a Scribe Transcription Service

This container must have access to the directories managed by the scribelicense container described above.

$ docker run --rm -d \
  --name="sqcserver" \
  -e PRODUCT_CLASS="[product_class]" \
  -p [target-port]:8888 \
  -v "$PWD/[license-dir]":/scribe/gktlicense \
  -v "$PWD/[gkt-folder]":/scribe/gkt \
  -v "$PWD/[db-directory]":/scribe/db \
  -v "$PWD/[data-directory]":/data \
  -e DATA_OUT="/data" \
  -e MODEL_TYPE="tradervoice" \
  -e USERS=2 \
  -e QUOTE_ACTIVATION_KEYWORD="confirm" \
  docker.greenkeytech.com/sqcserver

The credentials provided by GreenKey should include your scribe-licensekey along with its expiration date. Your target product_class is set at container runtime and limited by your license. Directories gktlicense and gkt are shared with the scribelicense container.

target-port is an open port on the machine that you will use to access the service.

db-directory is a directory where you store databases of common prices for your quotes and trades. These reference prices help our interpreters place decimal points correctly.

data-directory is a directory where you store captured quotes and trades for later processing.

model-type is either tradervoice (default) or general. Using the general model will give higher accuracy for general transcription but will disable quote / trade detection.

USERS is the maximum number of connections the container will allow (subject to license conditions).

An optional variable TRADE="True" will set the interpreter to interpret trades instead of quotes.

If you do not wish to store audio and JSON files, remove the DATA_OUT environmental variable from the docker run command.


Launching the Scribe Discovery Engine inside of the Scribe Dictation Server

SQCServer comes with a data extraction engine called Scribe Discovery Engine built-in. SQC uses this engine internally for capturing quotes from audio. If you are using SQCServer to capture quotes, then Discovery automatically starts up with SQCServer behind the scenes, and loads itself with the relevant quote interpreters for your license.

This same engine can be used to extract information according to user-created definiton files to create custom "skills." To use SQCServer and Discovery for custom skills, we must manually turn on Discovery by setting -e DISCOVERY="True" in the docker run command.

To get Discovery to interpret your custom skills, you must mount your custom definition files with the command -v "$PWD/[custom-directory]":/custom \.

Be sure to omit any configurations that tell SQC to look for quotes if you are using custom skills with Discovery, otherwise SQCServer will ignore your custom skills, and default to using built-in quote interpreters.

Below is a demo run script to use to launch SQCServer in conjunction with Discovery.

$ docker run --rm -d \
  --name="sqcserver" \
  -p 8888:8888 \
  -v "$PWD/data":/data \
  -v "$PWD/gktlicense":-d/scribe/gktlicense \
  -v "$PWD/gkt":/scribe/gkt \
  -e DATA_OUT="/data" \
  -e USERS=2 \
  -e DISCOVERY="True" \
  -e MODEL_TYPE="general" \
  -v "$PWD/[custom-directory]":/custom \
  docker.greenkeytech.com/sqcserver

Note that we have intentionally omitted PRODUCT_CLASS, QUOTE_ACTIVATION_KEYWORD, and we set MODEL_TYPE="general".




Step 5: Confirm the container is running

docker logs sqcserver




Step 6: Transcribe a File (Optional)

Post against a websocket address using a python script available here by specifying the websocket-ip and target-port. If you are licensed for interpreting European government bonds, this test file will demonstrate the expected output.

$ python client.py -u ws://[websocket-ip]:[target-port]/client/ws/speech quote_detection.mp3
BKO 12/18 -- / .645
It's, a quote like dec 18 schatz 64.5 offered I should see it populated the end of this object.

Should the service prove unavailable, a message will be returned indicating the status.


If you specified a data-directory when launching the container, then you will find two files inside of this directory after running the previous script. Both files are labelled with a UUID. The file that ends in .json is the full output response from SQCServer, and the other is the raw audio file that was submitted. (The UUID used in the following example output is for demonstration purposes only, the actual UUID for your files will be different.)

$  ls [data-directory]/
bda967d1-e849-43df-b8ae-8e2df8d89178.json  bda967d1-e849-43df-b8ae-8e2df8d89178.wav

Here is the output of the JSON response from SQCServer for the previously run command (truncated for brevity). Your output should resemble these contents. Note that if your model type is general, you will only have transcript outputted.

$ cat [data-directory]/bda967d1-e849-43df-b8ae-8e2df8d89178.json
{
  "status": 0,
  "segment": 0,
  "result": {
    "hypotheses": [
      {
        "interpreted_quote": {
          "imString": "BKO 12/18 -- / .645",
          "product": "BKO",
          "terms": ["12/18"],
          "bid": "",
          "ask": ".645",
          "structure": "basis"
        },
        "transcript": "if i say a quote like dec eighteen schatz sixty four and a half offered i should see it popular at the end of this object",
        "product_class": "egbs",
      }
    ]
    "is_quote": true
  }
}

Occasionally, a quote is misclassified by SQCServer and misinterpreted. This can happen when keywords are common across multiple product classes. To force SQCServer to classify a quote as egbs, you can manually specify the interpreter to use at the command line using python client.py --interpreter egbs -u ws://[websocket-ip]:[target-port]/client/ws/speech quote_detection.mp3.

If you are using the Scribe Discovery Engine, you can specify your domains and interpreters here using python client.py --domains [comma-separated-domain-names] --interpreters [comma-separated-interpreter-names]. Both parameters are optional, and if you fail to provide any interpreters, but do specify one or more domains, we will find the appropriate interpreter within those domains and interpret it in Discovery.




Step 7: Finalize your deployment

After setting up SQCServer, there are several options to connect to it.


1) Make a direct websocket connection

Our javascript example deployment shows you how to make a direct websocket connection to your server and receive the output in a browser.

Download the files above, then make sure the folling lines are uncommented in index.html and sqc/controller.js:

index.html

<!-- Begin Direct Connection Example -->
<button id="gktStart" onclick="toggleListening()">Start</button>
<button id="gktStop" onclick="toggleListening()">Stop</button>
<span id="gktServer" address="ws://[sqc-server-address]/client/ws/"></span>
<span id="gktRecorderWorker" address="sqc/recorderWorker.js">
<!-- End Direct Connection Example -->

Change the [sqc-server-address] to match that of your deployment.

sqc/controller.js

//// Direct connection example
function transcriptionReceived(code, data) {
  try {
    dataObject = JSON.parse(data);
    if ('result' in dataObject) {
      $('#JSONoutput').html(JSON.stringify(dataObject, null, 2));

            if ('hypotheses' in dataObject.result) {
                if ('clean_transcript' in dataObject.result.hypotheses[0]) {
                    $('#transcriptOutput').text(dataObject.result.hypotheses[0].clean_transcript)
                } else {
                    $('#transcriptOutput').text(dataObject.result.hypotheses[0].transcript)
                }
            }
    }
  }
  catch (err) {
    console.log(data);
  }
}

Once you get it running, it should look something like this:


NOTE: If you set up SQC with Discovery to output custom JSON payloads, the Javascript example will not work without modification. You will have to alter the files sqc/controller.js and sqc/dictate.js.

In sqc/controller.js, comment out the function transcriptionReceived. Below that function, you will see code that says //// Direct connection example with Cusom JSON from Discovery. Uncomment the following function, and replace the expectedKey variable with some key that you expect to come back from Discovery according to your custom JSON definition in schemas.json.

In sqc/dictate.js, comment out the block of code under the comment that reads // use the following code for default setup; comment it out for Custom JSON output from Discovery. Uncomment the block of code under the comment // use this code for Custom JSON output from Discovery. Replace the expectedKey variable with some key that you expect to come back from Discovery in this file as well.


2) Relay results over a websocket

Deploy the websocket relay server found in our call bot to enable the results to be broadcast to a backend endpoint. Once deployed, you can uncomment the following example in our javascript example deployment to relay the results received on the front-end via a second websocket connection that your application backend can listen to.

index.html

<!-- Begin Relay Example -->

PIN:<br>
<input type="text" id="gktPin"></input>
<button id="gktStart" onclick="toggleListening();readSocket()">Start</button>
<button id="gktStop" onclick="toggleListening()">Stop</button>
<span id="gktServer" address="ws://[sqc-server-address]/client/ws/"></span>
<span id="gktWSSServer" address="ws://[ws-server-address]/"></span>
<span id="gktRecorderWorker" address="sqc/recorderWorker.js">

<!-- End Relay Example -->

sqc/controller.js

//// Relay connection example
function transcriptionReceived(code, data) {
  try {
    dataObject = JSON.parse(data);
    if ('result' in dataObject) {
        socket.send(JSON.stringify({token: "XC7WFVC3UW3HPQRV5YUR", message: dataObject, endpoint: "/" + $('#gktPin').val()}));
    }
  }
  catch (err) {
    console.log(data)
  }
}

function readSocket() {
  var msg_socket = new WebSocket($('#gktWSSServer').attr('address') + $('#gktPin').val());

  msg_socket.onopen = function() {
    console.log("connected");
  }

  msg_socket.onclose = function(e) {
    console.log("connection closed (" + e.code + ")");
  }

  msg_socket.onmessage = function(e) {
    if (e.data.length > 0) {
        console.log("received message: " + e.data);
      $('#JSONoutput').html(JSON.stringify(JSON.parse(e.data), null, 2));
    }
  }

}

socketInit();

Make a websocket connection to your.websocket.address/pin that you entered to receive results.

NOTE: Some modern browsers (like Chrome) require a user gesture before allowing microphone access. This behavior varies from browser to browser, as well as within containers. In both of the above examples, you may see a warning like this in the developer console:

The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu

If this is the case, you must click the initialize button first, then click "Start" to start transcription.


3) Transcribe a SIP stream

Follow our call bot instructions to capture quotes and trades or transcribe in real-time via a SIP telephony connection.



SQCServer Versioning

We maintain tagged versions of all Scribe services with major, minor, and incremental version numbers x.y.z. The latest tag should always point to the most recent version, which is currently 3.2.12.

Presently, the following SQCServer versions are available on our docker repo: 3.2.9, 3.2.8 3.2.7, 3.2.6, 3.2.5, 3.2.4, 3.2.3, 3.2.2, 3.2.12, 3.2.11, 3.2.10, 3.2.0, 3.0.12