top of page
Getting Started
Install the magic-moonshine Python package.
TERMINAL
#
pip install magic-moonshine
Configure Moonshine with your API token, found in the developer panel.
PYTHON
1
2
3
import moonshine
moonshine.config(API="YOUR TOKEN")
Making a video index
You need to create a video index before uploading files to it.
PYTHON
1
moonshine.create("test-index-1")
Uploading to the index
To upload a file, specify both its source location and the destination index.
Moonshine will automatically assign a unique file ID in the index to prevent naming conflicts. This unique file ID is essential for making queries, so save it for future reference.
After uploading, videos can take several minutes to complete indexing and become searchable.
PYTHON
1
2
3
4
5
def progress_callback(progress: float):
print(progress)
fileid = await moonshine.upload("./video.mp4", "test-index-1", progress_callback)
# fileid = "FHXYU838JHDWK.mp4"
Searching through an index
Search through an index using any natural language query. Results are categorized into three types: embeddings, transcripts, and OCR (optical character recognition).
PYTHON
1
2
3
4
5
6
7
# Search with text
resp = moonshine.search(index="test-index-1", query="red car with people shaking hands inside")
# Search with image
resp = moonshine.search(index="test-index-1", image="./image.png")
# resp = {'status': 'complete', 'output': {'embeddings': match[], 'transcript': match[], 'ocr': match[]}}
Every match object is an individual result clip that matches the query. A list of match objects is returned for every result category.
A match object is a list with the following data:
0
1
2
3
4
Moonshine assigned video fIle name
Example: 'FHXYU838JHDWK.mp4'
Match Frame Start
Example: 660
Match Frame End
Example: 725
Match Timestamp Start
Example: '0 min, 22 sec'
Match Timestamp End
Example: '0 min, 24 sec'
Analyze an index
Answer open-ended questions about videos in the index. Context from multiple videos is automatically gathered to best generate a response.
A query is meant to be short; for best results, limit it to under 15 words. Queries are limited to 100 characters. If your project requires more context or guiding, use the guidance parameter explained below.
PYTHON
1
2
3
resp = moonshine.inquire("test-index-1", "How has the car changed over time?")
# resp = "The car was moved several times and sustained a dent on its left side when parked in the
# garage."
Advanced Parameters
Adding Guidance
For moonshine.inquire() calls that require external knowledge bases, you can add textual context, detailed instructions, or output formats, up to 500k tokens in length.
PYTHON
1
2
guidance = "Respond in the format: [{timestamp: float, description: string, video_id: string}...]"
resp = moonshine.inquire("test-index-1", "How has the car changed over time?", guidance=guidance)
Ground answers with a VideoTarget
By default, moonshine.inquire() uses context from all videos in an index to generate its response. If you want to focus the inquiry on a specific video and timestamp, you can provide a VideoTarget. While the inquire call will still consider context from all other media, it prioritizes the information from the specified VideoTarget in its response.
VideoTarget
(class)
Params
file_id
required
String
timestamp
required
Float
Description
Moonshine generated FileID of the target video created during upload
Example: FHXYU838JHDWK.mp4
Target timestamp of video, in seconds
Example: 30
PYTHON
1
2
target = moonshine.VideoTarget(file_id="FHXYU838JHDWK.mp4", timestamp=30)
resp = moonshine.inquire("test-index-1", "What caused the car to end up like this?", target=target)
bottom of page