Skip to content

Quickstart

1. Install coactive

Request a demo to gain access to the Coactive platform. Once you've been granted access, download our latest sdk and install it in your local python environment

pip install coactive_sdk/python/coactive-py3.whl

2. Authenticate

Next, load the authentication credentials necessary for calling the coactive APIs. Note that these variables are environment specific and will be provided by Coactive.

1
2
3
4
5
6
7
8
import coactive
from auth_credentials import COACTIVE_HOST, CLIENT_ID, CLIENT_SECRET

access_token = f"{CLIENT_ID}:{CLIENT_SECRET}"
configuration = coactive.Configuration(
    host = COACTIVE_HOST,
    access_token = access_token,
)

3. Initialize a dataset on coactive

To get started, provide coactive access to your visual data. The code below provides access to your data lake and creates a dataset on coactive. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
from coactive.apis import DatasetApi
from coactive.model.create_dataset_request import CreateDatasetRequest
from coactive.model.storage_type_enum import StorageTypeEnum

with coactive.ApiClient(configuration) as api_client:

    create_dataset_request = CreateDatasetRequest(
        name="visual_data",
        description="A dataset containing images and videos.",
        storage_type=StorageTypeEnum("s3"),
        data_path="s3://coactive-demo-datasets/quickstart/",
    )

    try:
        api_response = DatasetApi(api_client).create_dataset(create_dataset_request)
    except coactive.ApiException as e:
        print("Exception when calling DatasetApi: %s\n" % e)

dataset_id = api_response.dataset_id
Once coactive has access to your data lake, it will convert your unstructured visual data to semantically meaningful representations. This enables fast access to all your visual data and the core functionalities of coactive, such as intelligent search and real-time analytics.

The code below provides additional metadata about the semantic representations of your data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
from coactive.apis import EmbeddingApi

with coactive.ApiClient(configuration) as api_client:
    try:
        api_response = EmbeddingApi(api_client).get_embeddings_by_dataset_id(dataset_id)
    except coactive.ApiException as e:
        print("Exception when calling EmbeddingApi: %s\n" % e)

embedding_ids = {emb.name: emb.embedding_id for emb in api_response['data']}
embedding_id = embeddings_ids['multimodal']

4. Search your images

You can use coactive's intelligent search to quickly find the visual assets you are looking for. coactive's APIs allow you to semantically search through millions of images, videos and any associated metadata using text and/or images as shown below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
from coactive.apis import SimilaritySearchApi

text_query = 'a green dress'
image_query = 'path/to/example_dress.jpg'
limit = 5 # Only top 5 results

with coactive.ApiClient(configuration) as api_client:
    try:

        # For text-only search
        if text_query and not image_query:
            search_response = SimilaritySearchApi(api_client).search_images_by_text_grouped(
                embedding_id=embedding_id,
                query=text_query,
                limit=limit)

        # For image-only search
        elif not text_query and image_query:
            search_response = SimilaritySearchApi(api_client).get_similar_images_grouped_semantic(
                embedding_id=embedding_id,
                file=open(image_query, 'rb'),
                limit=limit)

        # For multimodal search
        elif text_query and image_query:
            search_response = SimilaritySearchApi(api_client).get_similar_images_grouped_semantic(
                embedding_id=embedding_id,
                file=open(image_query, 'rb'),
                metadata_value=text_query,
                limit=limit)         

    except coactive.ApiException as e:
        print("Exception when calling SimilaritySearchApi: %s\n" % e)

search_result_image_ids = [
    result['items'][0]['coactive_image_id'] 
    for result in api_response['data']
]

5. Define a visual concept

You can define a custom visual concept, such as green_dress, by simply providing a few example images. These visual concepts define which images are considered green_dress and not green_dress. This binary classification is our definition of a visual concept.

In the code below, we create the concept green_dress concept from the images found in the intelligent search above.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from coactive.apis import ConceptApi
from coactive.model.create_concept_request import CreateConceptRequest

with coactive.ApiClient(configuration) as api_client:
    create_concept_request = CreateConceptRequest(
        name="green_dress",
        description="Images of green dresses like path/to/example_dress.jpg",
        embedding_id=embedding_id,
        positive_examples = search_result_image_ids)

    try:
        api_response = ConceptApi(api_client).create_concept(create_concept_request)
    except coactive.ApiException as e:
        print("Exception when calling ConceptApi: %s\n" % e)

While it may seem simple, there are nuances to each visual concept (e.g., is an oil painting of a green dress considered green_dress or not green_dress?). These nuances are often crucial and are rarely captured by off-the-shelf image classification solutions.

coactive allows you to quickly and seamlessly define visual concepts, giving you complete control over the definition (and nuances) of your visual taxonomy and classification.

Moreover, coactive allows you to efficiently update these visual concepts as your tasks or visual data change over time.

6. Run real-time queries over your visual data

coactive's real-time analytics engine allows you to query your unstructured visual data using SQL by providing a structured view of your visual data (i.e. rows = visual asset, columns = metadata).

Visual concepts combined with the standard capabilities of SQL can easily answer analytical questions spanning visual and structured data. No heavy lifting or deep technical expertise is required!

Here is how coactive can help you answer analytical questions about the green_dress concept created using standard SQL syntax:

  1. Are there more examples of green_dress?
1
2
3
4
5
query_1 = '''
    SELECT count(*)
    FROM coactive_table
    WHERE green_dress = 1
'''
  1. How many green_dress images were uploaded? Does this vary by category?
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
query_2 = '''
    SELECT 
        meta.category,
        count(*)
    FROM coactive_table ct
    LEFT JOIN metadata_df meta
        ON ct.image_id = meta.image_id
    WHERE ct.green_dress = 1
    GROUP BY meta.category
'''
  1. Is green_dress a viral/growing trend?
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
query_3 = '''
    SELECT 
        meta.upload_date,
        count(*)
    FROM coactive_table ct
    LEFT JOIN metadata_df meta
        ON ct.image_id = meta.image_id
    WHERE ct.green_dress = 1
    GROUP BY meta.upload_date
    ORDER BY meta.upload_date
'''

The code below shows how you can run any of the queries above.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
from coactive.apis import QueryApi
from coactive.model.query_request import QueryRequest

with coactive.ApiClient(configuration) as api_client:
    try:
        api_response = QueryApi(api_client).execute_query(
            QueryRequest(query=query_1, embedding_id=embedding_id))
    except coactive.ApiException as e:
        print("Exception when calling QueryApi: %s\n" % e)

query_id = api_response.query_id

Once the query has finished running, the code below shows how you can view the results as a pandas.DataFrame 

1
2
3
4
5
6
7
8
9
import pandas as pd

with coactive.ApiClient(configuration) as api_client:
    try:
        query_response = QueryApi(api_client).get_query_by_id(query_id).to_dict()
    except coactive.ApiException as e:
        print("Exception when calling QueryApi: %s\n" % e)

results_df =  pd.DataFrame([row['data'] for row in query_response['results']['data']])

7. Tutorials

See our tutorials for the fastest way to get started in our most common use cases:

Please refer to the documentation for the various ways to interact with the Coactive platform and view a full list of functionality.