GreenKey

Transcribing Files

Scribe can be used as a service to transcribe files on demand or for single transient transcription jobs. Each running instance of SVTServer can only transcribe one file at a time.

See the section on Scaling Scribe for information on how to launch a high-availability service to transcribe multiple files at once.




Checking the Status of the Service

Before you begin a transcription job, you should check the status of the service to make sure it is available:

$ curl -X GET localhost:5000/status
{
  "message": "No transcription jobs currently in progress.", 
  "status": 0
}




Asynchronous Transcription

Uploading files

For files smaller than the maximum upload size, you can upload the files to the container:

$ curl -X POST localhost:5000/upload \
  -H "Content-type: multipart/form-data" \
  -F 'data={};type=application/json' \
  -F "file=@path/to/myfile.wav"

path/to/myfile.wav is the local path to the file from the directory the command is run from.

To increase the maximum upload size, see the section on synchronous transcription.

Note that uploaded file extensions are currently limited to the following formats: 'ogg', 'wav', 'mp3', 'pcm', 'aiff', 'aac', 'wma', 'm4a', 'flac', 'alac', 'g722', 'gsm'

Transcribing mounted files

For files larger than the maximum upload size, place the files in the storage-directory that was specified during the launch of the container. Then, tell the service to transcribe your file:

$ curl -X POST localhost:5000/ \
  -H "Content-type: application/json" \
  -d '{"file":"/files/myfile.wav"}'

/files/myfile.wav is the path of the file relative to the container based on the directory where it was mounted. The example shows an instance where a directory such as /home/someuser/myfiles was mounted as the storage-directory and myfile.wav was placed in it.


Obtaining final transcripts

If you uploaded your file, the final transcript will be available in the upload-directory that was specified at container launch.

If you stored your file in the storage-directory that was specified at container launch, the final transcript will be available in the same directory.

In both cases, the file extension will be .json with the same base file name as the original audio submitted.

The final transcript can also be obtained via the /async route:

$ curl -X GET localhost:5000/async/[filename]

filename is the name of the file without the extension




Synchronous Transcription

Synchronous transcription is useful for small files that take a short amount of time to transcribe.

Because the connection is maintained while a synchronous transcription job executes, the connection can eventually timeout. By default, synchronous transcription requests will timeout after 3600 seconds.

You can configure Scribe with a longer timeout through the following container configuration:

docker run \
  ...
  -e SYNC_TIMEOUT="7200" \
  ...


Uploaded files

For files smaller than the maximum upload size (default 20MB), you can upload the files to the container and obtain synchronous transcripts:

$ curl -X POST localhost:5000/sync/upload/[filetype] \
  -H "Content-type: multipart/form-data" \
  -F 'data={};type=application/json' \
  -F "file=@path/to/myfile.wav"

path/to/myfile.wav is the local path to the file from the directory the command is run from. [filetype] is an optional parameter describing which file type is returned (see Output Types).

Note that uploaded file extensions are currently limited to the following formats: 'ogg', 'wav', 'mp3', 'pcm', 'aiff', 'aac', 'wma', 'm4a', 'flac', 'alac', 'g722', 'gsm'

The request will block until the transcript is finished and returned. If the connection times out, an error message is returned to try again later. If this occurs, you can retry the sync request using the sync route for stored files below.

The maximum upload size can be configured as shown below.

NOTE The maximum upload size should only be increased from the default value if a mounted directory to store uploaded files has also been specified. If you do not do this, then the container can grow in size to a point where it will exceed system limits on access to resources and crash.


Global configuration

The value for MAX_UPLOAD_SIZE is in megabytes.

$ docker run \
  ...
  -e MAX_UPLOAD_SIZE="100" \
  ...


Stored files

For files larger than the maximum upload size, place the files in the storage-directory that was specified during the launch of the container. Then, tell the service to transcribe your file:

$ curl -X POST localhost:5000/ \
  -H "Content-type: application/json" \
  -d '{"file":"/files/myfile.wav"}'

/files/myfile.wav is the path of the file relative to the container based on the directory where it was mounted. The example shows an instance where a directory such as /home/someuser/myfiles was mounted as the storage-directory and myfile.wav was placed in it.

Then, wait until the transcription job finishes and retrieve the json output:

$ curl -X GET localhost:5000/file/myfile.json




Single Transcription Jobs

Scribe allows you to transcribe single files without launching a service. The file must be in the storage-directory for this to work, and the KEEP_ALIVE flag is removed. Other configuration options are detailed here.

With internet access:

$ docker run --rm \
  -e GKT_API="https://scribeapi.greenkeytech.com/" \
  -e GKT_USERNAME="[scribe-username]" \
  -e GKT_SECRETKEY="[scribe-secretkey]" \
  -v $(pwd)/[storage-directory]:/files \
  -e PROCS=2 \
  -e TARGET_FILE="/files/myfile.wav" \
  docker.greenkeytech.com/svtserver

Without internet access:

$ docker run --rm \
  -e LICENSE_KEY="[scribe-licensekey]" \
  -v $(pwd)/[storage-directory]:/files \
  -e PROCS=2 \
  -e TARGET_FILE="/files/myfile.wav" \
  docker.greenkeytech.com/svtserver

The credentials provided by GreenKey should include your scribe-username and scribe-secretkey, or your scribe-licensekey along with its expiration date.

storage-directory is a directory where the TARGET_FILE exists and where the final transcripts will appear.

PROCS is the number of CPU cores on your machine to use. Specify more cores for faster transcription.

TARGET_FILE is the path of the file relative to the storage-directory.




Output Types

In addition to JSON lattices as shown above, additional output types are available. These file types can be specified via environmental variable for single transcription jobs via the TYPE environmental variable. Alternatively, these file formats can be enforced by using /sync/upload/[filetype] routes when [filetype] is set to json, transcript, text, or stm.


(1) WebVTT Transcript (filetype transcript)

WEBVTT 

1 
00:00:00.000 --> 00:00:04.500 
<v Channel 1> testing channel one one two three four five <v Channel 2> testing channel two five six seven eight nine


Global configuration

Outputted files will have the extension .vtt instead of .json. HTTP requests will still return the JSON lattice.

$ docker run \
  ...
  -e TYPE="transcript" \
  ...


Obtaining the last final transcription

$ curl -X GET localhost:5000/transcript


(2) Text (filetype text)

testing channel one one two three four five testing channel two five six seven eight nine


Global configuration

Outputted files will have the extension .txt instead of .json. HTTP requests will still return the JSON lattice.

$ docker run \
  ...
  -e TYPE="text" \
  ...


Obtaining the last final transcription

$ curl -X GET localhost:5000/text


(3) STM (filetype stm)

filename 1 UnkSpeaker 0.00 2.19 <o,f0,male> testing channel one 
filename 2 UnkSpeaker 0.00 6.12 <o,f0,male> testing channel two 
filename 1 UnkSpeaker 2.18 8.62 <o,f0,male> one two three four five 
filename 2 UnkSpeaker 6.11 8.62 <o,f0,male> five six seven eight nine


Global configuration

Outputted files will have the extension .stm instead of .json. HTTP requests will still return the JSON lattice.

$ docker run \
  ...
  -e TYPE="stm" \
  ...


Callbacks on completion

You can specify a callback URL and required headers (comma separated) if you wish to receive a notification when a transcription job is done. Start the container with the following environmental variables:

$ docker run \
  ...
  -e CALLBACK_URL="http://callback.url/route" \
  -e CALLBACK_HEADERS="Authorization: Basic sometoken,Content-type: application/json" \
  ...

Your callback route will receive responses of the following format, with success status of done and a failure status of busy or failure.

{"status": "done", "file": "/uploads/comcast.wav"}

busy indicates the service was not available to process the job, while failure indicates the service did process the job but encountered an invalid file.


Emailing files after completion

With either the synchronous route or asynchronous route above, both transcripts and audio files can be sent to an email address. This can either be specified at runtime or on container launch, and it requires both a target email address and a SendGrid API key.

$ docker run \
  ...
  -e EMAIL_ADDRESS="user@example.com" \
  -e SENDGRID_KEY="[hash]" \
  ...

Via the /upload route:

$ curl -X POST localhost:5000/upload \
  -H "Content-type: multipart/form-data" \
  -F 'data={"emailAddress": "user@example.com","sendgridKey": "[hash]"}; type=application/json' \
  -F "file=@test.wav"

where [hash] is a SendGrid API key.