11-877: Advanced Topics in MultiModal Machine Learning (Fall 2022) - Carnegie Mellon University vs SavirOS

Teaches architectural patterns for combining visual, audio, and textual modalities through cross-modal attention mechanisms, transformer-based fusion layers, and late/early/hybrid fusion strategies. Covers implementation of joint embedding spaces where heterogeneous data types are projected into shared representational spaces, enabling downstream tasks like visual question answering and video understanding through coordinated feature alignment.

Unique: Structured curriculum from Carnegie Mellon's MultiComp Lab combining theoretical foundations with hands-on implementation of state-of-the-art fusion strategies (early fusion via concatenation, late fusion via score aggregation, hybrid attention-based fusion) with explicit coverage of alignment losses and contrastive learning objectives

vs alternatives: More comprehensive than generic deep learning courses by focusing exclusively on multimodal-specific architectures and fusion patterns, with direct access to CMU researchers' latest work rather than textbook-only material

Teaches design patterns for vision-language models (VLMs) including CLIP-style contrastive learning, image-text matching objectives, and transformer-based architectures that align visual and textual representations. Covers implementation of dual-encoder systems with shared embedding spaces, training strategies using contrastive losses (InfoNCE), and inference patterns for zero-shot classification and image-text retrieval.

Unique: Provides structured breakdown of CLIP-style architectures with explicit coverage of dual-encoder design, contrastive loss formulation (InfoNCE with temperature scaling), and inference-time optimization patterns for efficient similarity computation across large image databases

vs alternatives: Deeper technical treatment of vision-language alignment than general multimodal courses, with focus on the mathematical foundations of contrastive objectives and practical implementation details for production-scale systems

Teaches design patterns for transformer-based multimodal models including vision transformers (ViT) for image encoding, text transformers for language understanding, and cross-attention mechanisms that enable interaction between modalities. Covers architectural choices like shared vs separate token spaces, positional encoding strategies for different modalities, and training techniques (masked language modeling, masked image modeling, contrastive learning) adapted for multimodal transformers.

Unique: Detailed coverage of transformer-based multimodal architectures including vision transformer (ViT) design with patch embeddings, cross-attention mechanisms for modality interaction, and multimodal pre-training objectives (masked language modeling, masked image modeling, contrastive learning) adapted for transformer-based models

vs alternatives: More focused on transformer-specific multimodal design patterns than general multimodal architecture courses, with emphasis on attention mechanisms and pre-training strategies specific to transformer models

Teaches temporal modeling approaches for video understanding including 3D CNNs (C3D), two-stream networks (spatial + temporal pathways), and transformer-based video encoders. Covers how to capture motion patterns through optical flow, frame sampling strategies, and temporal attention mechanisms that learn which frames are semantically important for action recognition and video classification tasks.

Unique: Systematic coverage of temporal modeling paradigms including 3D convolutions with learnable temporal kernels, two-stream networks with explicit optical flow computation, and temporal segment networks that sample frames hierarchically to balance computational cost with temporal coverage

vs alternatives: More thorough treatment of temporal modeling than general computer vision courses, with explicit comparison of 3D CNN vs two-stream vs transformer approaches and their computational trade-offs

Teaches methods for learning and leveraging audio-visual synchronization, including cross-modal self-supervised learning where audio and video streams are used to supervise each other without labeled data. Covers synchronization detection (determining if audio and video are temporally aligned), audio-visual source separation (isolating individual speakers from mixed audio using visual cues), and learning joint representations through contrastive objectives that maximize agreement between aligned modalities.

Unique: Focuses on leveraging natural audio-visual synchronization as a self-supervision signal through contrastive learning (maximizing similarity between aligned audio-video pairs while minimizing similarity to misaligned pairs), with explicit coverage of source separation using visual information to guide audio decomposition

vs alternatives: Unique emphasis on audio-visual synchronization as a learning signal rather than treating audio and visual modalities independently, enabling self-supervised pre-training without manual annotations

Teaches methods for building retrieval systems that match queries in one modality (e.g., text) to candidates in another modality (e.g., images) using learned similarity metrics. Covers embedding-based retrieval where both modalities are projected into a shared space, ranking objectives like triplet loss and contrastive losses, and efficient indexing strategies (approximate nearest neighbor search) for scaling to millions of candidates while maintaining sub-second query latency.

Unique: Comprehensive treatment of embedding-based retrieval with explicit coverage of ranking objectives (triplet loss, contrastive losses, margin-based losses), efficient indexing via approximate nearest neighbor search (FAISS, LSH), and strategies for handling scale (millions of candidates) while maintaining sub-second latency

vs alternatives: More focused on cross-modal retrieval specifics than general information retrieval courses, with emphasis on metric learning for aligning heterogeneous modalities rather than single-modality ranking

Teaches principles of learning joint representations where different modalities are mapped into a shared embedding space that captures semantic relationships. Covers self-supervised learning objectives (contrastive, masked modeling), alignment losses that encourage modality-specific encoders to produce compatible embeddings, and evaluation metrics for measuring the quality of learned representations (downstream task performance, retrieval metrics, linear probe accuracy).

Unique: Systematic treatment of multimodal representation learning with explicit coverage of alignment objectives (InfoNCE, triplet loss variants), modality-specific encoder design, and evaluation protocols that measure both representation quality (linear probe accuracy) and downstream task transfer performance

vs alternatives: Deeper focus on multimodal-specific representation learning than general self-supervised learning courses, with emphasis on alignment between heterogeneous modalities rather than single-modality contrastive learning

Teaches architectures and training strategies for visual question answering (VQA) systems that combine visual understanding with natural language reasoning. Covers attention mechanisms that identify relevant image regions for answering questions, fusion of visual features with question embeddings, and training objectives that handle multiple correct answers and answer frequency bias. Includes coverage of VQA datasets (VQA v2, GQA) and evaluation metrics (accuracy, BLEU, CIDEr).

Unique: Comprehensive treatment of VQA architectures including spatial attention (identifying relevant image regions), channel attention (weighting feature maps), and fusion strategies for combining visual and textual information, with explicit coverage of handling answer frequency bias through weighted loss functions

vs alternatives: More specialized than general vision-language courses by focusing specifically on VQA task design, evaluation protocols, and known dataset biases that affect model performance

SavirOS is an AI-powered Relationship Operating System that enhances meeting preparation by auto-generating intelligence briefs, tracking promises, and compiling relationship memory, ensuring users are always prepared and informed for their meetings.

Unique: SavirOS uniquely compounds relationship intelligence across all interactions, making it smarter with each meeting unlike competitors that treat meetings in isolation.

vs alternatives: SavirOS offers a more integrated and intelligent approach to meeting preparation compared to traditional tools that focus solely on transcription or note-taking.

SavirAI is a triage-RAG agent that answers questions about relationships, schedules actions, drafts emails, generates documents, and manages contacts — all through natural conversation. 84 tools across 7 agents: platform, calendar, relationship, pre-meeting, post-meeting, communication, creation. Autonomy policy gates sensitive actions (email sending, rescheduling) behind user confirmation.

Seven AI-powered generators for meeting-related communications: icebreaker conversation starters, meeting agenda generator, follow-up email drafts, email subject line optimizer, meeting decline message writer, introduction email generator, and out-of-office reply creator. All free, no signup required.

Automatically enriches contacts with LinkedIn profile data (Proxycurl), company intelligence (Hunter.io), recent news (NewsData.io), and web search (Tavily). Creates comprehensive contact profiles with career history, company details, mutual connections, and recent activity.

Four utility tools: QR code generator (URL, WiFi, vCard, text — PNG/SVG export), browser-based image compressor (JPEG/PNG/WebP, no upload), JSON formatter/validator with tree view, and file sharing (up to 50MB, shareable links). All free, no signup, privacy-first.

Four free lookup tools: reverse caller ID (global, spam detection, confidence scoring), professional email finder (Hunter.io verification), person lookup (career history, talking points via Proxycurl/Tavily), and company lookup (industry, funding, team size, news, social links).

Five meeting utilities: real-time meeting timer with agenda tracking, meeting link decoder (extracts ID/passcode from Zoom/Teams/Meet URLs), instant meeting link generator, WhatsApp link builder with prefilled messages, and downloadable .ics calendar event creator.

Auto-detects ended meetings (every 3 minutes). Processes transcripts from Recall.ai, Fireflies.ai, or user-pasted notes. Extracts structured summary, key points, decisions (with rationale and decision maker), and commitments. Builds episodic memory records. Extracts individual facts and consolidates into per-contact intelligence profiles.