Alterra.ai FAQ search and chatbot API

Free-text search for FAQs, help centers, knowledge bases, user guides, canned support responses…

A chatbot that answers straightforward questions. If the answer is in the FAQ file the bot shall be able to answer it.

Using this API you may build your own conversational virtual agents, messenger bots, embed this functionality in site search or sales or support workflow, pair it up with live agents, etc.

This API performs natural-language search against a set of FAQ articles.

An article contains a title (aka canonical question) and full-text answer. An article may also have a short snippet attached to it, to be shown where the dialog space is constrained (think small chat windows or text messages).

You also have an option to load a set of question paraphrases and assign the correct answers to them. Together, they form the training corpus for the AI. This greatly benefits the quality of future searches.

The system keeps a log of queries it receives. Past queries may also be assigned correct answers, and added to the training corpus.

The API has four main parts:

  1. performing search,
  2. uploading and editing the FAQ articles,
  3. working with the training corpus and query log,
  4. managing FAQ settings

The first part is used during the serving time. It performs actual search. The user asks a question or enters a search query; you pass the query to this API, and it returns the search results, ordered by relevancy.

The other three parts are used ahead of time, to upload and edit data, and train machine learning.

The API may return more than one search result. If your application is a fully automated bot you would display only the first result. If it is site search you may display several results. If you have a human agent in the loop you may display several results to the agent and let him manually select which one to send to the end-user.

Minimally, you would upload FAQ articles, then use the Search API in your message flow.

All APIs define here follow the REST paradigm.

All methods require an API key. API key is passed as “Authorization” header in the request.

All GET methods take arguments as CGI parameters in the URL.

All POST and PUT requests take arguments in the request body, which should be in JSON format.

All methods return JSON. It may be empty if the only result is an operation status which is reported as HTTP status code.

Entities

Each entity is represented by a JSON object.

Article

One article (question-answer pair) from the FAQ document

Field name Type Always present Description
id int Y
question string Y Title of the article, aka canonical question
answer string Y Full answer
alt_answers list of strings N Other formulations of the answer
snippet string N Short summary of the answer
action string N Action to be taken in responce to user inquiry
comment string N Private comment, not visible to end-users

Strings may contain HTML formatting.

Normally, only answer is displayed to end-users. alt_answers (optional) would be logically equivalent to answer, but expressed in different words. They may be sometimes shown instead of the main answer, to make the bot appear “less robotic”.

Action is the name of a method to be called in responce to the user inquiry, in addition to or instead of displaying the answer. Examples of actions include: send email or message, display a new webpage, create a support ticket, create a task in task management software or CRM, etc.

Example:

{
    "id": 42,
    "question": "Where can I find more bots?",
    "snippet": "There is no official bot store yet",
    "answer": "There is no official bot store at the moment, so you‘ll have to ask your friends or search the web for now. We’re pretty sure you'll find some bots to play with."
}

Query

Users ask questions in their own words. Many of these questions (queries) are logically equivalent and shall be answered by the same answer. They are all paraphrases of one canonical question (question). These paraphrases (referred to as “queries”) are stored in the system and used for training AI.

These queries may be imported from the log of actual queries entered by actual users (see Log Entry), or from other sources (e.g. public training corpora). They may be even thought up by you – think how you could rephrase the canonical question.

Field name Type Always present Description
text string Y Query’s text
hash string N A hash of this query’s text, used as its ID
article_id int Y id of the FAQ article that answers this query

article_id shall be trusted; it is assigned by a human labeler and used for training the algorithm.

A special value of -1 is used when there is no good answer in the FAQ

Example:

{
    "text": "Where's the money Lebowski?",
    "hash":  1138,
    "article_id": -1
}

Together, Articles and Queries form the training corpus for Machine Learning.

Log entry

An actual user query that user entered into the system in the past.

All or some of these user queries may be exported to the system and become a part of the training corpus. Other user queries may be pre-filtered as irrelevant, garbage, spam, etc. and ignored. Thus, not all queries from the raw query log belong to the training corpus.

Field name Type Always present Description
query_text string Y User query text
search_id string Y The ID of the search assigned when the search was performed
query_hash string Y Normalized query text hash
timestamp datetime Y Date and time in ISO 8601 (RFC 3339) format
article_ids list of ints N List of article_ids returned by the search algorithm as the response to this query, in the order of relevancy (see Search Response)

Unlike article ids in Queries, these article ids cannot be trusted. They are assigned by the search algorithm and may contain errors.

Example:

{
    "query_text": "How do one discover new bots?",
    "query_hash": "F95C6BB30EC32F55",
    "search_id": "05F4F53B-4F5D8162-7852A351-4B90F22E",
    "timestamp": "2017-03-29T12:00:35Z",
    "article_ids": [42, 17, 5]
}

Search response

List of all results returned by the search engine in response to given search query, ordered by relevancy (in websearch it would be called SERP)

Field name Type Always present Description
search_id string Y The ID of the search
query_hash string Y Normalized query text hash
timestamp datetime Y Date and time in ISO 8601 (RFC 3339) format
results list of search result objects Y List of search results, ordered by relevancy

Example:

{
  "search_id": "05F4F53B-4F5D8162-7852A351-4B90F22E",
  "query_hash": "F95C6BB30EC32F55",
  "timestamp": "2017-03-29T12:00:35Z",
  "results": [
    {
      "article_id": 42,
      "question": "Where can I find new bots",
      "snippet": "In your internets",
    },
    {
      "article_id": 17,
      "question": "Where can I find new boots",
      "snippet": "In a shop",
    },
    {
      "article_id": 5,
      "question": "Where can I find new boats",
      "snippet": "On a wharf",
    },
  ]
}

Search result

One result of FAQ search, represented by one FAQ article.

Field name Type Always present Description
article_id int Y Article ID of the result
question string Y Article title (aka canonical question)
snippet string N A short snippet of the article text
answer string N Full article text
action string N Action to be taken in responce to user inquiry

Snippet and answer may contain HTML formatting.

Example:

{
    "article_id": 42,
    "question": "Where can I find more bots?"
}

FAQ settings

These settings allow you to control the algorithm and data (the training corpus). The algorithm control parameters may include the number of search results returned by the algorithm, the relevancy cut-off, etc. (not yet available.)

The data part allows you to combine your main FAQ corpus with pre-defined common corpora. The latter may include dialogs related to chitchat (smalltalk), your business address, driving directions, business hours, accepted forms of payment, etc. User queries related to these topics are rather universal, and this API has a collection of such common questions. (The answers may be specific to your business — you would edit them accordingly.) You may add these common corpora to your main FAQ.

Currently, there is only one such common corpus, called “chitchat”. It contains dialogs that would enable your bot to answer smalltalk questions like “how are you?”, “are you a bot or human?”, etc. More common corpora will be added in the future.

FAQ settings

Object describing all settings for this API. Currently, it contains the list of common FAQs merged with your main corpus.

Field name Type Always present Description
merge_in list of merged FAQ objects N Common corpora merged with your corpus

Merged FAQ

Name of the common FAQ corpus merged in your corpus.

Field name Type Always present Description
name string Y Name of the merged common FAQ corpus

Common FAQ

A common FAQ available to merge into your corpus.

Field name Type Always present Description
name string Y Name of common FAQ
description string N Human-readable description of the corpus

Search API

This is the main part. It is used during the serving time. It performs actual search. The user queries your application, you pass the query to this API, which returns the search results, ordered by relevancy.

Given a user query, find relevant answer(s) in the FAQ: GET from /api/faq/v1/search

Arguments

Field name Type Required Description
query string Y user query

Example: /api/faq/v1/search?query=How+can+I+contact+you

Server reply

A search response object (list of search results, ordered by relevancy)

Example:

{
  "search_id": "73b61636-29b1-4bee-8845-3bc0ffe9a86a",
  "results":
    [
        {
            "article_id": "42",
            "question": "Where can I find more bots?"
        },
        {
            "article_id": "43",
            "question": "Who can see me ‘online’?",
            "snippet": "People can only see you online if you're sharing your last seen status with them."
        }
    ]
}

Articles API

This part is used for uploading and editing the FAQ articles.

Add new articles: POST to /api/faq/v1/articles/

Upload new article(s) to the system.

Arguments

Field name Type Always present Description
body list of article objects Y new articles to upload

Example:

"body": [
    {
        "question": "Where can I find more bots?",
        "snippet": "There is no official store yet",
        "answer": "There is no official store at the moment, so you‘ll have to ask your friends or search the web for now. We’re pretty sure you'll find some bots to play with.",
        "id": "42"
    },
    {
        "question": "Who can see me ‘online’?",
        "answer": "People can only see you online if you're sharing your last seen status with them. There is one exception to this: people will be able to see you online for a brief period when you send them a message in a one-on-one chat or in a group where you both are members.",
        "id": "54"
    }
]

Server reply

List of identifiers of just uploaded articles.

If the articles were uploaded with specific idsthen these ids are honored. If the articles were uploaded without ids then the system automatically assigns new ids to them. ids of all just uploaded articles are returned here.

Example:

[42,54]

Update existing articles: PUT to /api/faq/v1/articles/

Edit (replace) existing article(s).

Arguments

Field name Type Always present Description
body list of article objects Y data for articles you are updating

Example:

"body": [
    {
        "id": 42,
        "question": "Where can I find more bots?",
        "snippet": "There is no official store yet",
        "answer": "There is no official store at the moment, so you‘ll have to ask your friends or search the web for now. We’re pretty sure you'll find some bots to play with."
    },
    {
        "id": 54,
        "question": "Who can see me ‘online’?",
        "answer": "People can only see you online if you're sharing your last seen status with them. There is one exception to this: people will be able to see you online for a brief period when you send them a message in a one-on-one chat or in a group where you both are members."
    }
]

Server reply

List of ids for replaced articles (as a confirmation).

Example:

[42,54]

Delete one article: DELETE /api/faq/v1/articles/{article_id}

Server reply

Appropriate HTTP status

Get a list of articles: GET from /api/faq/v1/articles/

Export articles from the system. It could be used to retrieve all articles, or a portion of (with pagination).

Arguments

Field name Type Required Description
offset int N Offset in the sorted list of articles. May be negative (Then is applied from the end of the list) Default value is 0
limit int N Maximum number of articles to return. Default value is 10

Example: /api/faq/v1/articles?offset=42&limit=2

Server reply

List of article objects

Example:

[
    {
        "article_id": 42,
        "question": "Where can I find more bots?",
        "answer": "There is no official store at the moment, so you‘ll have to ask your friends or search the web for now. We’re pretty sure you'll find some bots to play with."
    },
    {
        "article_id": 43,
        "question": "Who can see me ‘online’?",
        "snippet": "People can only see you online if you're sharing your last seen status with them.",
        "answer": "People can only see you online if you're sharing your last seen status with them. There is one exception to this: people will be able to see you online for a brief period when you send them a message in a one-on-one chat or in a group where you both are members."
    }
]

Get one article: GET from /api/faq/v1/articles/{article_id}

Export one specific article, identified by article_id, from the system.

Server reply

An article object or an appropriate HTTP status and error message

Example:

{
    "article_id": 42,
    "question": "Where can I find more bots?",
    "answer": "There is no official store at the moment, so you‘ll have to ask your friends or search the web for now. We’re pretty sure you'll find some bots to play with."
}

Queries API

This part is used for working with the query log and training corpus for Machine Learning.

There are two types of queries:

  1. All queries entered by users in the past – see Log entries
  2. Queries admitted to the training corpus – see Queries

This API deals with the latter.

These two sets have a big overlap by may be not equal. Indeed, only legitimate user queries shall be added to the training corpus. Garbage and spam shall be discarded. On the other hand, some queries in the training corpus may come from sources other than logged user queries (e.g. other public training corpora).

You should invoke re-training of the ML models after finishing edting the training corpus with this API.

Update queries: POST to /api/faq/v1/queries/

Add new queries to the training corpus or update existing queries.

Only unique queries will be actually added. Existing queries will be replaced. The result will contain the list of query hashes corresponding to the given list of queries. Duplicate queries will have the same hashes. article_id shall be determined by human labelers and must be set in all queries.

Arguments

Field name Type Required Description
body list of Query objects Y New queries to upload

article_id field must be set in all queries

Example:

"body": [
    {
        "text": "Who should I contact about my booking?",
        "article_id": 2
    },
    {
        "text": "Where is my confirmation?"
        "article_id": 5
    },
    {
        "text": "WHERE IS MY CONFIRMATION???"
        "article_id": 5
    },
]

Server reply

List of respective query hashes.

Example:

["67F227A57F1A496F", "47E25DF1D9BAF663", "47E25DF1D9BAF663"]

Delete one query: DELETE /api/faq/v1/queries/{query_hash}

Server reply

Appropriate HTTP status

Get a list of queries: GET from /api/faq/v1/queries/

Export query objects from the training corpus (with pagination). It could be used to retrieve all queries, or a portion of.

Arguments

Field name Type Required Description
offset int N Offset in the sorted list of queries. May be negative (Then is applied from the end of the list)
limit int N Number of queries to return. Default value is 100

Example: /api/faq/v1/queries?offset=1138&limit=1

Server reply

List of query objects

Example:

[
    {
        "hash": "23db4",
        "text": "Who should I contact about my booking?",
        "article_id": 2
    }
]

Get one query: GET from /api/faq/v1/queries/{query_hash}

Export one specific query, identified by hash, from the training corpus.

Server reply

A query object

Example:

{
    "hash": "34512",
    "text": "Where is my confirmation?",
    "article_id": 13
}

API for queries attached to FAQ articles

The queries for which the correct article is known with certainty can be retrieved and managed through this additional API which is article-based. These queries are “attached” to the appropriate article.

Update article’s queries: POST to /api/faq/v1/articles/{article_id}/queries/

Attach queries to a specific article. If given queries are attached to other articles, they are detached and attached to the selected article.

Arguments

Field name Type Required Description
body list of Query objects N Queries to attach

hash and article_id are ignored.

Example:

[
    {
    "text": "Where is my booking confirmation?"
    }
]

Server reply

List of respective query hashes

Example:

["706CB0285892CF2E"]

Delete one query attached to the article: DELETE /api/faq/v1/articles/{article_id}/queries/{query_id}

Server reply

Appropriate HTTP status

Get list of queries attached to the article: GET from /api/faq/v1/articles/{article_id}/queries/

Arguments

Field name Type Required Description
offset int N zero-based
limit int N default value 100. Max value 1000

Example: /api/faq/v1/articles/1/queries?offset=1138&limit=2

Server reply

List of query objects

Example:

[
    {
    "hash": "123e4",
    "article_id": 1,
    "text": "Where's the money Lebowski?"
    },
    {
    "id": "9ffc94",
    "article_id": 1,
    "text": "Who should I contact about my booking?"
    }
]

Get one query attached to the article: GET from /api/faq/v1/articles/{article_id}/queries/{query_hash}

Retrieve one specific query, identified by the query_hash

Server reply

query object

Log API

This part is used for working with the raw log of all queries entered by end-users in the past – see Log entries. The set of queries in this log may not fully coincide with the Queries in the training corpus. Indeed, only legitimate user queries shall be added to the training corpus. Garbage and spam may be discarded.

The intended use of this API is as follows. The system logs all end-user queries. You retrieve them, one-by-one or in bulk, make humans assign the right canonical answer (article id) to each query, add them to the training set (via Queries API), and invoke re-training of the ML models. You may use dedicated (hired) labelers or rely on the end-user or community feedback.

For you, the raw query log is read-only. The system logs all end-user queries, as is. You may terieve them, but not modify.

Get list of log entries: GET from /api/faq/v1/log/

Export log entry objects from the the raw query log (with pagination). It could be used to retrieve the entire query log, or a portion of.

Arguments

Field name Type Required Description
offset int N Offset in the list of log entries (sorted from oldest to newest). May be negative (Then it is applied from the end of the list, thus taking N newest entries)
limit int N default value 10. Max value 1000

Example: /api/faq/v1/log?timestamp=2017-03-21&offset=4238&limit=1

Server reply

List of log entry objects

Example:

[
    {
        "query": "How can I pay?",
        "search_id": "73b61636-29b1-4bee-8845-3bc0ffe9a86a",
        "query_hash": "01C5A0BBD64963CC",
        "timestamp": "2017-03-21T13:23:53+00:00",
        "result_ids": [ 3, 5, 8 ]
    }
]

Get one log entry: GET from /api/faq/v1/log/{search_id}

Retrieve one specific log entry, identified by the search_id.

Server reply

Log entry object

Example:

{
    "query": "How can I pay?",
    "search_id": "73b61636-29b1-4bee-8845-3bc0ffe9a86a",
    "query_hash": "01C5A0BBD64963CC",
    "timestamp": "2017-03-21T13:23:53+00:00",
    "result_ids": [ 3, 5, 8 ]
}

Train Machinde Learning API

This API invokes re-training of ML algorithms and re-indexing data, after completing editing the training corpus.

Train Machine Learning: POST to /api/faq/v1/train

Re-train ML algorithms and re-index data.

Arguments

None

Server reply

Appropriate HTTP status

FAQ settings API

These API allows you to combine your main FAQ corpus with pre-defined common corpora.

Currently, there is only one such common corpus, called “chitchat”. It contains dialogs that would enable your bot to answer smalltalk questions like “how are you?”, “are you a bot or human?”, etc. More common corpora will be added in the future.

Get FAQ settings: GET from /api/faq/v1/faq_settings

Retrieve FAQ settings

Server reply

FAQ settings object

Example:

{
    "merge_in": [
        { "name": "chitchat" }
    ]
}

Upload new FAQ settings: POST to /api/faq/v1/articles/faq_settings

Arguments

Field name Type Required Description
body FAQ settings object Y New settings

Example:

{
    "merge_in": [
        { "name": "chitchat" }
    ]
}

Server reply

Appropriate HTTP status

Get list of common FAQ corpora: GET from /api/faq/v1/commonFaqs

Get a list of all available common FAQs you can merge into your own corpus.

Arguments

Does not take any arguments

Server reply

A list of common FAQ objects

Example:

[{
    "name": "chitchat",
    "description": "Small talk / chitchat corpus"
}]