Media Handling

Cryptid handles media (profile pictures, file attachments, stickers) without requiring server-side storage. This architecture preserves privacy, reduces server complexity, and gives users full control over their data.

Core Principles:

Small media: Thumbnails stored inline for instant display
Large media: Device-to-device transfer on-demand
No server storage: Servers never see or store media content
End-to-end encrypted: All media encrypted with MLS-derived or per-file keys
User control: Clients decide retention policies and availability
Client caching: Media cached locally for instant future access

Profile Pictures

Profile pictures use a hybrid approach: small thumbnails for instant display, full resolution via device-to-device transfer.

Structure

struct ProfilePicture {
    // Thumbnail for instant preview (in InfoPackages/bundles)
    thumbnail: Vec<u8>,       // AVIF, 64x64, max 16kb

    // Full image via device-to-device transfer
    file_id: String,          // Format: {device_id}-profile-{timestamp}
    size: u64,
    encryption_key: [u8; 32], // Random key for decryption
    mime_type: String,        // Must be "image/avif"
}

Requirements

Format:

AVIF only (required for consistency)
Square aspect ratio (enforced)
Lossless compression mode

Size Limits:

Thumbnail: Max 16KB (64x64 pixels)
Full image: Max 200 KB (512x512 pixels recommended)

Validation: Implementations MUST reject profile pictures that:

Exceed size limits
Use non-square aspect ratios
Use formats other than AVIF

Upload Flow

use image::DynamicImage;

fn set_profile_picture(image: DynamicImage) -> Result<ProfilePicture> {
    // 1. Enforce square aspect ratio
    let size = image.width().min(image.height());
    let squared = image.crop_imm(0, 0, size, size);

    // 2. Generate thumbnail (64x64, AVIF lossless)
    let thumb = squared.resize_exact(64, 64, FilterType::Lanczos3);
    let thumb_data = encode_avif_lossless(&thumb)?;

    if thumb_data.len() > 16_000 {
        return Err("Thumbnail exceeds 16 KB");
    }

    // 3. Encode full image (512x512 max, AVIF lossless)
    let full = squared.resize_to_fill(512, 512, FilterType::Lanczos3);
    let full_data = encode_avif_lossless(&full)?;

    if full_data.len() > 200_000 {
        return Err("Profile picture exceeds 200 KB");
    }

    // 4. Store locally with unique file_id
    let file_id = format!("{}-profile-{}", device_id, current_unix_timestamp());
    store_file_locally(&file_id, &full_data)?;

    // 5. Generate random encryption key
    let encryption_key = random_32_bytes();

    Ok(ProfilePicture {
        thumbnail: thumb_data,
        file_id,
        size: full_data.len() as u64,
        encryption_key,
        mime_type: "image/avif".to_string(),
    })
}

Download Flow

use image::DynamicImage;

async fn fetch_profile_picture(
    pic: &ProfilePicture,
    owner_device: DeviceId,
) -> Result<DynamicImage> {
    // 1. Display thumbnail immediately
    display_image(&pic.thumbnail)?;

    // 2. Request full image via device-to-device
    let file_request = MessageContent::FileRequest {
        file_id: pic.file_id.clone(),
        chunk_offset: 0,
        chunk_size: pic.size,
    };

    send_to_device(owner_device, file_request).await?;

    // 3. Receive FileData response
    let encrypted_data = receive_file_data().await?;

    // 4. Decrypt with provided key
    let plaintext = decrypt(&encrypted_data, &pic.encryption_key)?;

    // 5. Decode and display
    decode_avif(&plaintext)
}

File Attachments

File attachments use device-to-device transfer with MLS-derived encryption keys. They’re announced via MessageContent::FileAttachment and transferred via FileRequest/FileData messages.

Announcement

Senders announce files using MessageContent::FileAttachment:

FileAttachment {
    filename: String,           // Original filename
    mime_type: String,          // e.g., "image/jpeg", "application/pdf"
    size: u64,                  // Size in bytes

    plaintext_hash: [u8; 32],   // Blake3 of unencrypted content
    content_hash: [u8; 32],     // Blake3 of encrypted content

    file_id: String,            // Format: {sender_device_id}-{random_u64}

    caption: Option<String>,    // Optional caption text
}

Field Specification:

filename: No path separators. UTF-8 required. Client sanitizes.
mime_type: Hint only. No validation or enforcement.
size: Informational. Clients use for progress and storage checks.
plaintext_hash: Blake3 of unencrypted file. Used for deduplication and verification.
content_hash: Blake3 of encrypted file.
file_id: Unique identifier for this file on sender’s device.
caption: Optional text accompanying the file (e.g., “Here’s the contract”).

Encryption

Files are encrypted using keys derived from MLS group epoch secret:

fn derive_encryption_key(
    mls_epoch_secret: &[u8],
    plaintext_hash: &[u8; 32],
) -> [u8; 32] {
    hdkf_expand(
        mls_epoch_secret,
        b"file-key",
        plaintext_hash,
    )
}

Why MLS-derived keys?

Cross-group isolation: Same file in different groups = different encryption keys
- Alice sends photo to Group A and Group B
- Group A key: HKDF(epoch_A, plaintext_hash)
- Group B key: HKDF(epoch_B, plaintext_hash)
- These yield different ciphertexts, so the server cannot correlate.
Forward secrecy: Removed members cannot decrypt new files
- When Bob leaves, group gets new epoch secret
- Bob cannot derive new encryption keys
- Files uploaded after removal are inaccessible to Bob
Deterministic key derivation: Same file = same key (for deduplication)
- Clients can identify files by plaintext_hash
- Know they’re the same content across groups
- Cache locally without re-fetching

Upload Flow

Sender encrypts file with MLS epoch-derived key
Sender stores encrypted file locally with file_id
Sender sends CryptidMessage with MessageContent::FileAttachment
Message routed through MLS group (encrypted again with MLS)
Recipients receive file announcement

Download Flow

Recipient receives FileAttachment message
Recipient sends MessageContent::FileRequest to sender
Sender responds with one or more MessageContent::FileData messages
Recipient decrypts each chunk using MLS epoch secret
Recipient verifies plaintext_hash matches
Recipient caches locally for future use

File Transfer Protocol

Request a file:

FileRequest {
    file_id: String,   // Which file to request
    chunk_offset: u64, // Byte offset (for resumable downloads)
    chunk_size: u64,   // Requested chunk size
}

Send file data:

FileData {
    file_id: String,      // Which file this data belongs to
    chunk_offset: u64,    // Byte offset of this chunk
    data: Vec<u8>,        // Encrypted chunk (base64 in JSON)
    is_final_chunk: bool, // True if this completes the file
}

Chunking

Files larger than 1 MB SHOULD be transferred in chunks.

Recommended chunk size: 512 KB

Benefits:

Resumable downloads (if interrupted, resume from chunk_offset)
Memory efficiency (no full file in memory)
Progress indicators for users

Example:

async fn send_file_chunked(
    file_id: &str,
    recipient: DeviceId,
    mls_group: &MlsGroup,
) -> Result<()> {
    let file_data = load_file_locally(file_id)?;
    let chunk_size = 512 * 1024; // 512 KB

    for (idx, chunk) in file_data.chunks(chunk_size).enumerate() {
        let is_final = (idx + 1) * chunk_size >= file_data.len();

        send_message(recipient, MessageContent::FileData {
            file_id: file_id.to_string(),
            chunk_offset: (idx * chunk_size) as u64,
            data: chunk.to_vec(),
            is_final_chunk: is_final,
        }).await?;
    }

    Ok(())
}

Epoch Handling

When MLS epoch changes (member added/removed):

Old files remain encrypted with old epoch keys
Clients MUST retain epoch history to decrypt old files
Recommended retention: 30 days or 50 epochs (whichever is longer)
New files encrypted with new epoch key

Decryption with epoch history:

fn decrypt_file_from_group(
    ciphertext: &[u8],
    metadata: &FileAttachment,
    mls_group: &MlsGroup,
) -> Result<Vec<u8>> {
    // Try current epoch first
    let current_secret = mls_group.export_secret(b"file-encryption", 32);
    let key = derive_file_encryption_key(&current_secret, &metadata.plaintext_hash);

    if let Ok(plaintext) = chacha20poly1305_decrypt(&key, ciphertext) {
        return Ok(plaintext);
    }

    // Try historical epochs
    for epoch in mls_group.epoch_history() {
        let epoch_secret = epoch.export_secret(b"file-encryption", 32);
        let key = derive_file_encryption_key(&epoch_secret, &metadata.plaintext_hash);

        if let Ok(plaintext) = chacha20poly1305_decrypt(&key, ciphertext) {
            return Ok(plaintext);
        }
    }

    Err("Could not decrypt with any epoch key")
}

Availability

Files are available only while sender device is online and retains the file.

Retention policy (client-enforced):

Recommended: 30 days from upload
User-configurable: 7 days to indefinitely
Auto-delete after inactivity period

Offline recipients:

Queue FileRequest messages
Fetch when sender comes online
No server-side queuing

Stickers

Groups can have sticker packs stored in MLS group extensions. Stickers are full message reactions (unlike inline custom emoji).

Structure

// Stored in group extension: "custom_media"
struct GroupCustomMedia {
    packs: HashMap<String, MediaPack>,
    emojis: HashMap<String, MediaItem>, // Inline emoji (see below)
}

struct MediaPack {
    // Unique identifier (e.g., "pack-alice-cats")
    pack_id: String,
    // Display name (e.g., "Alice's Cats")
    name: String,
    // Creator
    uploaded_by: DeviceId,
    uploaded_at: u64,
    items: HashMap<String, MediaItem>,
}

struct MediaItem {
    // Item ID within pack or emoji ID
    id: String,

    // Small items: stored inline in group metadata
    inline_data: Option<InlineMedia>,

    // Large items: device-to-device transfer
    file_reference: Option<FileReference>,
}

struct InlineMedia {
    // "image/avif", "image/webp"
    mime_type: String,
    // Max 50 KB
    data: Vec<u8>,
}

struct FileReference {
    // Device-to-device file ID
    file_id: String,
    size: u64,
    mime_type: String,
}

Size Tiers

Tier	Size Range	Storage Method	Use Case
Small	0-50KB	Inline in group metadata	Static stickers
Medium	50-500KB	Device-to-device transfer	Animated stickers
Large	> 500KB	Rejected from group metadata	Client should compress

Adding Packs

Group members with appropriate permissions can add packs:

async fn add_sticker_pack(
    &self,
    group_id: &GroupId,
    pack: MediaPack,
) -> Result<()> {
    // Check permission
    let group = self.groups.get(&group_id)?;
    let perm_ext = group.get_permission_extension()?;
    let my_perms = perm_ext.device_permissions
      .get(&self.device.device_id)
      .ok_or("Device not found in group permissions")?;

    if !my_perms.contains(Permissions::MANAGE_MEDIA_PACKS) &&
       !my_perms.contains(Permissions::ADMINISTRATOR) {
      return Err("Insufficient permissions to add shared media");
    }

    // Validate sizes
    for item in pack.items.values() {
        if let Some(inline) = &item.inline_data {
            if inline.data.len() > 50_000 {
                return Err("Inline item exceeds 50 KB");
            }
        }

        if let Some(file_ref) = &item.file_reference {
            if file_ref.size > 500_000 {
                return Err("File reference exceeds 500 KB");
            }
        }
    }

    // Update group extension
    let mut custom_media = group.extensions.custom_media;
    custom_media.packs.insert(pack.pack_id.clone(), pack);

    group.update_extension("custom_media", custom_media).await?;
    Ok(())
}

Using Stickers

Sending a stickers:

MessageContent::Sticker {
    pack_id: String,
    sticker_id: String,
    fallback: Option<Vec<u8>>, // Optional thumbnail preview
}

Display flow:

async fn display_sticker(
    sticker_msg: &Sticker,
    group: &Group,
) -> Result<()> {
    // 1. Look up pack in group metadata
    let pack = group.extensions.custom_media
        .packs.get(&sticker_msg.pack_id)?;

    let item = pack.items.get(&sticker_msg.sticker_id)?;

    // 2. Check if inline
    if let Some(inline) = &item.inline_data {
        display_image(&inline.data)?;
        return Ok(());
    }

    // 3. Show fallback thumbnail while fetching
    if let Some(fallback) = &sticker_msg.fallback {
        display_image(&fallback)?;
    }

    // 4. Request from uploader via FileRequest
    if let Some(file_ref) = &item.file_reference {
        let data = request_file(pack.uploaded_by, &file_ref.file_id).await?;
        display_image(&data)?;
    }

    Ok(())
}

Availability

Inline stickers: Always available (stored in group metadata)
Device-to-device stickers: Require uploader online
Clients SHOULD cache fetched stickers locally for future use

Custom Emoji

Custom emoji are inline in Text messages using the :emoji_id: format. They’re stored in the group’s custom_media extension as small media items.

Structure

struct CustomEmoji {
    // Emoji identifier (e.g., "neofoxsmile", "blobfoxlaugh")
    id: String,
    // Always inline (max 50 KB)
    inline_data: InlineMedia,
}

struct InlineMedia {
    // "image/avif", "image/webp"
    mime_type: String,
    // Compressed emoji, max 50 KB
    data: Vec<u8>,
}

Adding Emoji

async fn add_custom_emoji(
    group_id: GroupId,
    emoji_id: String,
    image_data: Vec<u8>,
) -> Result<()> {
     // Validate size
    if image_data.len() > 50_000 {
        return Err("Emoji exceeds 50 KB");
    }

    // Update group extension
    let mut custom_media = group.extensions.custom_media;
    custom_media.emojis.insert(emoji_id.clone(), CustomEmoji {
        id: emoji_id,
        inline_data: InlineMedia {
            mime_type: "image/avif".to_string(),
            data: image_data,
        },
    });

    group.update_extension("custom_media", custom_media).await?;
    Ok(())
}

Using in Text

Sender sends:

{
    "type": "Text",
    "data": {
        "text": "Looks great! :thumbs-up: Love it :party-blob:",
        "reply_to": null
    }
}

Client rendering:

fn render_text_with_emojis(
    text: &str,
    group: &Group,
) -> Result<RichText> {
     let custom_media = &group.extensions.custom_media;

    // Find all :emoji_id: patterns
    let re = Regex::new(r":([a-z0-9_-]+):")?;

    let rendered = re.replace_all(text, |caps: &Captures| {
        let emoji_id = &caps;[1]

        // Look up in custom emoji
        if let Some(emoji) = custom_media.emojis.get(emoji_id) {
            // Render emoji inline
            return "[emoji]".to_string();
        }

        // Not found, leave as-is
        format!(":{emoji_id}:")
    });

    Ok(RichText::from_string(rendered))
}

This should render as: “Looks great! 👍 Love it [party-blob]“

Benefits

Lightweight: Emoji format is just text substitution (:emoji_id:)
Backward compatible: Unknown emoji shows as :emoji_id: literally
No separate protocol: Lives entirely in Text messages
Always available: Stored inline in group metadata
Client flexibility: Clients render however they want

Server Enforcement

Maximum Message Size

Servers MUST enforce a maximum message size: 10 MB

Messages exceeding this limit are rejected with error 4003 MESSAGE_TOO_LARGE.

What Servers See

Observable:

Message size (possibly padded to 10 MB)
Routing metadata (recipient addresses)
Timestamp

Not observable:

Sender information
Media content (encrypted)
File types or names
Sticker/emoji usage
Whether message contains media or just text

No Server Storage

Servers do NOT store media. All media is either:

Inline in messages/metadata (encrypted)
Transferred device-to-device (never touches server)

Media Caching and Local Storage

Download-Once Strategy

Clients SHOULD implement a download-once, cache-forever strategy for all media:

First fetch: Request from sender device (or group metadata for inline media)
Verify: Check plaintext_hash and content_hash match
Store locally: Save to device storage with metadata
Reuse: All future accesses use cached copy
Retry only if missing: Fetch from sender only if local cache deleted/corrupted

Benefits:

Reduces load on sender devices (popular media fetched once)
Improves UX (instant display from cache)
Resilient to sender going offline
Reduces bandwidth usage significantly
Works fully offline after initial fetch

Cache Storage Structure

struct MediaCache {
    files: HashMap<[u8; 32], CachedFile>,

    // Sticker packs for quick access
    sticker_packs: HashMap<String, CachedStickerPack>,

    // Emoji cache (small, always loaded)
    emojis: HashMap<String, CachedEmoji>,
}

struct CachedFile {
    plaintext_hash: [u8; 32],
    plaintext: Vec<u8>,           // Decrypted content
    mime_type: String,
    size: u64,

    // Metadata for cache management
    cached_at: u64,
    last_accessed: u64,
    access_count: u32,

    // For cleanup/verification
    verified: bool,               // Hash verified
    is_favorite: bool,            // User-pinned (don't delete)
}

struct CachedStickerPack {
    pack_id: String,
    items: HashMap<String, CachedStickerItem>,
    cached_at: u64,
}

struct CachedStickerItem {
    sticker_id: String,
    data: Vec<u8>,
    mime_type: String,
    last_accessed: u64,
}

struct CachedEmoji {
    emoji_id: String,
    data: Vec<u8>,
    mime_type: String,
}

Downloading with Caching

async fn fetch_file_with_cache(
    plaintext_hash: &[u8; 32],
    file_ref: &FileAttachment,
    sender: DeviceId,
    mls_group: &MlsGroup,
) -> Result<Vec<u8>> {
    // 1. Check cache first
    if let Some(cached) = cache.files.get(plaintext_hash) {
        cached.last_accessed = now();
        cached.access_count += 1;
        return Ok(cached.plaintext.clone());
    }

    // 2. Not in cache, fetch from sender
    let encrypted = request_file_from_device(sender, &file_ref.file_id).await?;

    // 3. Decrypt using MLS epoch key
    let epoch_secret = mls_group.export_secret(b"file-encryption", 32);
    let encryption_key = derive_file_encryption_key(&epoch_secret, plaintext_hash);
    let plaintext = chacha20poly1305_decrypt(&encryption_key, &encrypted)?;

    // 4. Verify hashes
    if Blake3(&plaintext) != *plaintext_hash {
        return Err("Plaintext hash mismatch - corrupted or modified");
    }
    if Blake3(&encrypted) != file_ref.content_hash {
        return Err("Content hash mismatch - corrupted or modified");
    }

    // 5. Store in cache
    cache.files.insert(*plaintext_hash, CachedFile {
        plaintext_hash: *plaintext_hash,
        plaintext: plaintext.clone(),
        mime_type: file_ref.mime_type.clone(),
        size: plaintext.len() as u64,
        cached_at: now(),
        last_accessed: now(),
        access_count: 1,
        verified: true,
        is_favorite: false,
    });

    Ok(plaintext)
}

Cache Eviction Policy

Clients SHOULD implement cache cleanup based on:

Storage quota: Per-group or per-user storage limit
Access patterns: Remove least-recently-used (LRU) items
Age: Remove items older than retention period
User pinning: Never delete user-favorited media

Offline Access

Once media is cached locally:

Profile pictures display instantly
Stickers available in picker
Previously viewed files/images accessible
Custom emoji always available (inline)
No network request needed

Scenarios:

Device offline: Use cache for all media
Sender offline: Use cache (sender unreachable for fetch)
Network slow: Use cache immediately, background refresh if enabled

Cache Invalidation

Cache entries are invalidated when:

Hash mismatch: Corrupted file detected (never use again)
Manual deletion: User explicitly clears cache or removes item
Retention expired: Age exceeds policy retention period

Sender-initiated updates:

Sender deletes old profile picture and uploads new one
New file_id and plaintext_hash prevent cache collision
Clients fetch new version automatically

Best Practices

DO:

Cache all media immediately after fetch
Check cache before requesting from sender
Verify both plaintext_hash and content_hash
Store with access_count and last_accessed for analytics
Allow users to pin favorite media (never evict)
Show cache stats in settings (used storage, item count)

DON’T:

Re-fetch media if cached copy exists
Delete cache without user consent
Store unverified (hash-mismatched) media
Ignore corrupted cache entries

User Controls

Clients SHOULD expose cache management UI:

Settings options:

Cache storage quota
Retention period (7 days to indefinitely)
Eviction strategy preference
“Clear cache” button
“Pin/favorite” toggle per media item
View cache statistics (size, item count, oldest/newest)

Security Considerations

Privacy Properties

End-to-end encryption:

All media encrypted before transmission
Profile pictures: random encryption keys
File attachments: MLS-derived keys per group
Stickers/emoji: encrypted with MLS group key

Server blindness:

Server cannot see media content
Server cannot correlate files across groups
Server cannot track who downloads what

Cross-group isolation:

Same file in different groups = different ciphertext
Prevents correlation attacks
Server cannot build social graph from shared media

Denial of Service Mitigation

Per-device limits (client-enforced):

File retention: 30 days recommended
Storage quota: User-configurable
Auto-cleanup of old files

Message size limits (server-enforced):

Max 10 MB per message
Prevents bandwidth exhaustion

Chunked transfers:

Prevents memory exhaustion
512 KB chunks recommended

Forward Secrecy

MLS epoch-derived encryption:

Files encrypted with current epoch secret
Removed members cannot decrypt files uploaded after removal
Clients retain epoch history for decrypting old files
Recommended retention: 30 days or 100 epochs

Availability Tradeoffs

Device-to-device transfers require sender online:

Acceptable tradeoff for privacy
Users can manually re-share if sender unavailable
Thumbnails provide immediate preview
Inline media always available

Implementation Guidelines

Client Responsibilities

MUST:

Compress profile pictures to meet size/format requirements
Store advertised files locally with file_id mapping
Handle FileRequest messages from group members
Implement chunked transfers for files > 1 MB
Validate media sizes and formats before accepting
Retain MLS epoch history for file decryption (30 days minimum)
Cache all media locally after first fetch
Verify plaintext_hash and content_hash after decryption

SHOULD:

Check cache before requesting from sender
Implement progress indicators for chunked transfers
Provide retry logic for failed transfers
Auto-cleanup old files based on retention policy
Show cache statistics in settings
Pre-load stickers when group is opened

MAY:

Implement message padding for traffic analysis resistance
Allow user-configurable retention policies
Provide bandwidth/storage usage statistics
Sync cache across user’s devices

Server Responsibilities

MUST:

Enforce 10 MB max message size
Route FileRequest/FileData messages

MUST NOT:

Decrypt media
Store file content
Track file downloads
Correlate files across groups
Inspect message content
Store media files

Summary

Media Type	Max Size	Storage Method	Encryption	Always Available?
Profile Picture Thumbnail	16 KB	Inline in InfoPackage	Random key	✅ Yes
Profile Picture Full	200 KB	Device-to-device	Random key	⏳ When device online
File Attachments	No limit*	Device-to-device chunked	MLS epoch-derived	⏳ When device online
Stickers (inline)	50 KB	Group metadata	MLS group key	✅ Yes
Stickers (large)	500 KB	Device-to-device	MLS epoch-derived	⏳ When device online
Custom Emoji	50 KB	Group metadata	MLS group key	✅ Yes

*Practical limit: 10 MB per chunk, unlimited chunks