This page documents the v2 API. In v3, these managers are accessed directly on the MentraSession object (e.g. session.speaker, session.camera, session.led). See the MentraSession page for the current documentation, or the Migration Guide for all v2 → v3 changes.

Camera Module Reference

The Camera Module provides photo capture, managed streaming (cloud-orchestrated HLS/DASH/WebRTC), and unmanaged streaming (direct to your endpoint via SRT, RTMP, or WHIP).

Overview

Access the camera module through your app session:

const photo = await session.camera.requestPhoto();
const stream = await session.camera.startLivestream();
await session.camera.startLocalLivestream({ streamUrl: 'srt://example.com:4201?streamid=key' });

Photo Functionality

requestPhoto()

Request a photo from the connected smart glasses.

async requestPhoto(options?: PhotoRequestOptions): Promise<PhotoData>

Parameters

Parameter	Type	Description
`options`	`PhotoRequestOptions`	Optional configuration for the photo request

PhotoRequestOptions

interface PhotoRequestOptions {
  /** Whether to save the photo to the device gallery */
  saveToGallery?: boolean;
  /** Custom webhook URL to override the TPA's default webhookUrl */
  customWebhookUrl?: string;
  /** Authentication token for custom webhook authentication */
  authToken?: string;
  /** Desired photo size: "small" | "medium" | "large" | "full" */
  size?: "small" | "medium" | "large" | "full";
  /** Image compression level for upload optimization */
  compress?: "none" | "medium" | "heavy";
}

Compression Options

The compress option controls image compression during webhook upload to optimize network transfer speed. Defaults to "none" (no compression).

Level	Dimensions	JPEG Quality	Typical Size Reduction	Use Case
`"none"`	Original	Original	0% (no compression)	High-quality photos, fast WiFi
`"medium"`	75% scale	80% quality	~50-60% smaller	Balanced quality/speed
`"heavy"`	50% scale	60% quality	~75-85% smaller	Slow connections, large photos

Important Notes:

Compression only applies to webhook uploads (WiFi/cellular)
BLE transfers use a different ultra-aggressive compression (AVIF format, 10-15KB) regardless of this setting, controlled by the size parameter
Aspect ratio is always preserved
Uses JPEG format with bilinear scaling for high-quality results

Understanding Webhook vs BLE Compression

MentraOS uses two different compression algorithms depending on the transfer method: Webhook Compression (controlled by compress option):

Format: JPEG
Algorithm: Percentage-based dimension scaling + JPEG quality control
Use case: WiFi/cellular uploads where bandwidth is measured in Mbps
Result: 75-300KB files with good visual quality

BLE Transfer Compression (controlled by size option):

Format: AVIF (with JPEG fallback)
Algorithm: Fit-within-box dimension constraint + ultra-low quality AVIF encoding
Use case: Bluetooth fallback where bandwidth is ~50-100 KB/s
Result: 10-15KB files with acceptable quality for thumbnails

The system automatically selects the appropriate compression:

First attempts webhook upload (uses compress setting)
Falls back to BLE if webhook fails (uses aggressive AVIF compression based on size)

Returns

Returns a Promise<PhotoData> that resolves with the captured photo data.

PhotoData Interface

interface PhotoData {
  /** The actual photo file as a Buffer */
  buffer: Buffer;
  /** MIME type of the photo (e.g., 'image/jpeg') */
  mimeType: string;
  /** Original filename from the camera */
  filename: string;
  /** Unique request ID that correlates to the original request */
  requestId: string;
  /** Size of the photo in bytes */
  size: number;
  /** Timestamp when the photo was captured */
  timestamp: Date;
}

Example

// Basic photo request (no compression)
const photo = await session.camera.requestPhoto();
console.log(`Photo taken at timestamp: ${photo.timestamp}`);
console.log(`MIME type: ${photo.mimeType}, size: ${photo.size} bytes`);

// Access raw photo data
const photoBuffer = photo.buffer;
const base64Photo = photo.buffer.toString('base64');

// Save to gallery
const photoWithSave = await session.camera.requestPhoto({
  saveToGallery: true
});

// With medium compression (recommended for most use cases)
const compressedPhoto = await session.camera.requestPhoto({
  size: 'large',
  compress: 'medium'
});

// Custom webhook with authentication and compression
const photoWithWebhook = await session.camera.requestPhoto({
  customWebhookUrl: 'https://my-api.example.com/photo-upload',
  authToken: 'bearer-token-or-api-key',
  size: 'large',
  compress: 'medium'
});

Managed Streaming Functionality (Recommended)

Managed streaming provides zero-infrastructure streaming where the cloud handles ingest and returns HLS/DASH URLs for viewing. Multiple apps can access the same managed stream simultaneously.

startLivestream()

Start a managed stream with automatic URL generation.

async startLivestream(options?: ManagedStreamOptions): Promise<ManagedStreamResult>

Parameters

Parameter	Type	Description
`options`	`ManagedStreamOptions`	Optional configuration for the managed stream

ManagedStreamOptions

interface ManagedStreamOptions {
  /** Optional video configuration settings */
  video?: VideoConfig;
  /** Optional audio configuration settings */
  audio?: AudioConfig;
  /** Optional stream configuration settings */
  stream?: StreamConfig;
  /** Optional RTMP destinations to re-stream to (e.g., YouTube, Twitch).
   *  When present, switches from WebRTC to SRT ingest + HLS/DASH playback. */
  restreamDestinations?: { url: string; name?: string }[];
  /** Controls stream start/stop sounds. Defaults to true. */
  sound?: boolean;
}

ManagedStreamResult

interface ManagedStreamResult {
  /** HLS URL for viewing (functional in SRT mode, i.e. when restreamDestinations provided) */
  hlsUrl: string;
  /** DASH URL for viewing (functional in SRT mode, i.e. when restreamDestinations provided) */
  dashUrl: string;
  /** WebRTC (WHEP) URL for low-latency playback (functional in default WebRTC mode) */
  webrtcUrl?: string;
  /** Internal stream ID */
  streamId: string;
  /** Cloud-hosted preview/player URL */
  previewUrl?: string;
  /** Thumbnail image URL for previews */
  thumbnailUrl?: string;
}

Example

// Default: WebRTC for low-latency playback
const result = await session.camera.startLivestream();
console.log('WebRTC URL:', result.webrtcUrl);

// With restream destinations: SRT + HLS/DASH
const result = await session.camera.startLivestream({
  restreamDestinations: [
    { url: 'rtmp://a.rtmp.youtube.com/live2/YOUR-KEY', name: 'YouTube' }
  ],
  video: { frameRate: 30 },
  audio: { sampleRate: 48000 }
});
console.log('HLS URL:', result.hlsUrl);

stopLivestream()

Stop the current managed stream.

async stopLivestream(): Promise<void>

checkExistingStream()

Check if there’s an existing active stream (managed or unmanaged) for the current user.

async checkExistingStream(): Promise<StreamCheckResult>

StreamCheckResult Interface

interface StreamCheckResult {
  hasActiveStream: boolean;
  streamInfo?: {
    type: 'managed' | 'unmanaged';
    streamId: string;
    status: string;
    createdAt: Date;
    // Managed streams only:
    hlsUrl?: string;
    dashUrl?: string;
    webrtcUrl?: string;
    previewUrl?: string;
    thumbnailUrl?: string;
    activeViewers?: number;
    // Unmanaged streams only:
    streamUrl?: string;
    requestingAppId?: string;
  };
}

Example

const streamCheck = await session.camera.checkExistingStream();

if (streamCheck.hasActiveStream) {
  if (streamCheck.streamInfo?.type === 'managed') {
    console.log('HLS URL:', streamCheck.streamInfo.hlsUrl);
    // Join existing managed stream
    const result = await session.camera.startLivestream();
  } else {
    console.log('Active stream:', streamCheck.streamInfo?.streamUrl);
    console.log('Requested by:', streamCheck.streamInfo?.requestingAppId);
  }
} else {
  const result = await session.camera.startLivestream();
}

Managed Stream Status Monitoring

onLivestreamStatus()

Subscribe to managed stream status updates.

onLivestreamStatus(handler: (status: ManagedStreamStatus) => void): () => void

ManagedStreamStatus

interface ManagedStreamStatus {
  type: string;
  status: 'initializing' | 'preparing' | 'active' | 'stopping' | 'stopped' | 'error';
  hlsUrl?: string;
  dashUrl?: string;
  webrtcUrl?: string;
  message?: string;
  streamId?: string;
  timestamp: Date;
}

Example

const unsubscribe = session.camera.onLivestreamStatus((status) => {
  if (status.status === 'active') {
    console.log(`HLS URL: ${status.hlsUrl}`);
  } else if (status.status === 'error') {
    console.error(`Stream error: ${status.message}`);
  }
});

// Later, unsubscribe
unsubscribe();

Unmanaged Streaming Functionality

Unmanaged streaming sends the camera feed directly to your ingest endpoint. You choose the protocol via the URL scheme. Only one unmanaged stream can be active at a time, and it blocks other apps from using the camera.

startLocalLivestream()

Start an unmanaged stream to a specified URL.

async startLocalLivestream(options: StreamOptions): Promise<void>

Parameters

Parameter	Type	Description
`options`	`StreamOptions`	Configuration options for the stream

StreamOptions

interface StreamOptions {
  /** Stream URL. Supports srt://, rtmp://, rtmps://, and https:// (WHIP) */
  streamUrl: string;
  /** Optional video configuration settings */
  video?: VideoConfig;
  /** Optional audio configuration settings */
  audio?: AudioConfig;
  /** Optional stream configuration settings */
  stream?: StreamConfig;
  /** Play start/stop sounds (default: true) */
  sound?: boolean;
}

VideoConfig

interface VideoConfig {
  width?: number;      // pixels (e.g., 1280)
  height?: number;     // pixels (e.g., 720)
  bitrate?: number;    // bits per second (e.g., 2000000 for 2 Mbps)
  frameRate?: number;  // fps (e.g., 30)
}

AudioConfig

interface AudioConfig {
  bitrate?: number;          // bits per second (e.g., 128000 for 128 kbps)
  sampleRate?: number;       // Hz (e.g., 44100)
  echoCancellation?: boolean;
  noiseSuppression?: boolean;
}

StreamConfig

interface StreamConfig {
  durationLimit?: number; // max duration in seconds (e.g., 1800 for 30 min)
}

Example

// SRT stream
await session.camera.startLocalLivestream({
  streamUrl: 'srt://my-server.com:4201?streamid=my-key'
});

// RTMP stream
await session.camera.startLocalLivestream({
  streamUrl: 'rtmp://live.example.com/stream/key'
});

// WHIP (WebRTC) stream
await session.camera.startLocalLivestream({
  streamUrl: 'https://whip.example.com/ingest/endpoint'
});

// Advanced configuration
await session.camera.startLocalLivestream({
  streamUrl: 'srt://my-server.com:4201?streamid=key',
  video: { width: 1920, height: 1080, bitrate: 5000000, frameRate: 30 },
  audio: { bitrate: 128000, sampleRate: 44100 },
  stream: { durationLimit: 1800 }
});

stopLocalLivestream()

Stop the current unmanaged stream.

async stopLocalLivestream(): Promise<void>

Unmanaged Stream Status Monitoring

onLocalLivestreamStatus()

Subscribe to unmanaged stream status updates.

onLocalLivestreamStatus(handler: StreamStatusHandler): () => void

StreamStatusHandler

type StreamStatusHandler = (status: StreamStatus) => void;

StreamStatus

interface StreamStatus {
  type: string;
  streamId?: string;
  status: 'initializing' | 'connecting' | 'reconnecting' | 'streaming'
    | 'error' | 'stopped' | 'active' | 'stopping' | 'disconnected'
    | 'timeout' | 'reconnected' | 'reconnect_failed';
  errorDetails?: string;
  appId?: string;
  stats?: {
    bitrate: number;
    fps: number;
    droppedFrames: number;
    duration: number;
  };
  timestamp: Date;
}

Example

const unsubscribe = session.camera.onLocalLivestreamStatus((status) => {
  console.log(`Stream status: ${status.status}`);

  if (status.status === 'active' && status.stats) {
    console.log(`Bitrate: ${status.stats.bitrate} bps`);
    console.log(`FPS: ${status.stats.fps}`);
  } else if (status.status === 'error') {
    console.error(`Stream error: ${status.errorDetails}`);
  }
});

unsubscribe();

Unmanaged Stream Utility Methods

isCurrentlyStreaming()

isCurrentlyStreaming(): boolean

getCurrentStreamUrl()

getCurrentStreamUrl(): string | undefined

getStreamStatus()

getStreamStatus(): StreamStatus | undefined

Streaming Comparison

Feature	Managed Streaming	Unmanaged Streaming
Infrastructure Required	None	Your own ingest server
Protocols	Cloud-selected (SRT)	SRT, RTMP/RTMPS, WHIP
Multiple Apps Can Stream	Yes	No (Exclusive)
Blocks Other Apps	No	Yes
Viewer URLs Provided	HLS/DASH/WebRTC	You manage
Best For	Social media, prototypes, multi-app	Custom servers, exclusive access
Requires Internet	Yes	Depends on server location

Error Handling

Photo Errors

Timeout: Photo requests timeout after 30 seconds
Cancellation: Requests can be cancelled manually or during session cleanup
Device Errors: Camera unavailable or hardware issues

Stream Errors

Already Streaming: Cannot start while any stream (managed or unmanaged) is active
Invalid URL: URL must start with srt://, rtmp://, rtmps://, http://, or https://
Network Issues: Connection problems to ingest endpoint
Device Limitations: Hardware doesn’t support requested configuration

Getting Started

v3 (SDK 3.x)

v2 (Legacy)

​Camera Module Reference

​Overview

​Photo Functionality

​requestPhoto()

​Parameters

​PhotoRequestOptions

Compression Options

Understanding Webhook vs BLE Compression

​Returns

​PhotoData Interface

​Example

​Managed Streaming Functionality (Recommended)

​startLivestream()

​Parameters

​ManagedStreamOptions

​ManagedStreamResult

​Example

​stopLivestream()

​checkExistingStream()

​StreamCheckResult Interface

​Example

​Managed Stream Status Monitoring

​onLivestreamStatus()

​ManagedStreamStatus

​Example

​Unmanaged Streaming Functionality

​startLocalLivestream()

​Parameters

​StreamOptions

​VideoConfig

​AudioConfig

​StreamConfig

​Example

​stopLocalLivestream()

​Unmanaged Stream Status Monitoring

​onLocalLivestreamStatus()

​StreamStatusHandler

​StreamStatus

​Example

​Unmanaged Stream Utility Methods

​isCurrentlyStreaming()

​getCurrentStreamUrl()

​getStreamStatus()

​Streaming Comparison

​Error Handling

​Photo Errors

​Stream Errors

Camera Module Reference

Overview

Photo Functionality

requestPhoto()

Parameters

PhotoRequestOptions

Returns

PhotoData Interface

Example

Managed Streaming Functionality (Recommended)

startLivestream()

Parameters

ManagedStreamOptions

ManagedStreamResult

Example

stopLivestream()

checkExistingStream()

StreamCheckResult Interface

Example

Managed Stream Status Monitoring

onLivestreamStatus()

ManagedStreamStatus

Example

Unmanaged Streaming Functionality

startLocalLivestream()

Parameters

StreamOptions

VideoConfig

AudioConfig

StreamConfig

Example

stopLocalLivestream()

Unmanaged Stream Status Monitoring

onLocalLivestreamStatus()

StreamStatusHandler

StreamStatus

Example

Unmanaged Stream Utility Methods

isCurrentlyStreaming()

getCurrentStreamUrl()

getStreamStatus()

Streaming Comparison

Error Handling

Photo Errors

Stream Errors