Library

• model openai/whisper-large-v3

  Whisper

  OpenAI's large-scale speech recognition model, supports 99 languages with high accuracy.

  Used for: Speech-to-text, multilingual transcription

  • Askimo
  • VocalEats
  • ATC Communication
  • Educational Voice Assistant
• model timbrooks/instruct-pix2pix

  InstructPix2Pix

  Diffusion model for editing images via natural language instructions.

  Used for: Text-guided image editing and manipulation

  • Chimera Studio
• model stabilityai/stable-diffusion-xl-base-1.0

  Stable Diffusion XL

  Latest Stable Diffusion model with improved quality and coherence.

  Used for: High-quality text-to-image generation (1024×1024+)

  • Chimera Studio
• model mattmdjaga/segformer_b2_clothes

  SegFormer

  Efficient transformer-based segmentation model trained on apparel datasets.

  Used for: Semantic segmentation of clothing in images

  • Chimera Studio
• model lllyasviel/control-lora-canny-safetensors

  ControlNet

  Novel control mechanism for guiding image generation with edge/depth maps.

  Used for: Spatial control over diffusion model outputs

  • Chimera Studio
• model ai-forever/Real-ESRGAN

  Real-ESRGAN

  Practical generative adversarial network for real-world image enhancement.

  Used for: Image super-resolution and upscaling

  • Chimera Studio
• model sarvamai/OpenHathi

  Sarvam Models

  Indian language models for speech recognition and NLP, supporting Hindi and regional languages.

  Used for: Indian language NLP and voice

  • Askimo
• model mistralai/Mistral-7B-Instruct-v0.1

  Mistral 7B Instruct

  Efficient 7B parameter model optimized for instruction following and chat.

  Used for: General-purpose instruction-following language model

  • Educational Voice Assistant
  • Zenfinity
  • Askimo
  • VocalEats
  • MarketingHub
• model meta-llama/Llama-3.1-8B-Instruct

  Llama 3.1 8B Instruct

  Meta's latest open-weight LLM with strong reasoning and chat capabilities.

  Used for: High-quality instruction-following and reasoning

  • Zenfinity
  • RAG System for Clinics
  • MarketingHub
• model sentence-transformers/all-MiniLM-L6-v2

  Sentence Transformers (MiniLM)

  Lightweight sentence embeddings model for RAG and semantic search applications.

  Used for: Dense vector embeddings for semantic search and similarity

  • Educational Voice Assistant
  • RAG System for Clinics
• paper 1706.03762

  Attention Is All You Need

  Seminal paper introducing the Transformer architecture, basis for modern LLMs and foundation models.

  Used for: Foundation for Transformer architecture and attention mechanisms

  • Zenfinity
  • Educational Voice Assistant
• paper 2106.09685

  LoRA: Low-Rank Adaptation

  Parameter-efficient fine-tuning technique enabling model customization with minimal memory.

  Used for: Efficient fine-tuning of large language models

  • Zenfinity
  • Beastfluence
• paper 2203.02155

  InstructGPT / RLHF

  Introduces Reinforcement Learning from Human Feedback (RLHF) for model alignment.

  Used for: Alignment of language models through human feedback

  • Zenfinity
• paper 2212.04356

  Whisper: Robust Speech Recognition via Large-Scale Weak Supervision

  OpenAI's work on training speech models from weak supervision at scale.

  Used for: Multilingual speech-to-text foundation models

  • Askimo
  • VocalEats
  • ATC Communication
• paper 2211.09800

  InstructPix2Pix: Learning to Follow Image Editing Instructions

  Diffusion model trained to follow natural language instructions for image editing.

  Used for: Text-guided image editing and synthesis

  • Chimera Studio
• paper 2005.11401

  Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

  Introduces RAG pattern for augmenting language models with external knowledge.

  Used for: Combining retrieval and generation for factual grounding

  • Educational Voice Assistant
  • RAG System for Clinics
  • MarketingHub
• paper 2307.01952

  SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

  Stability AI's improvements to diffusion for larger images with better detail.

  Used for: High-resolution and high-fidelity image generation

  • Chimera Studio
• paper 2006.11239

  DDPM: Denoising Diffusion Probabilistic Models

  Seminal paper on diffusion-based generative modeling.

  Used for: Foundational diffusion model theory

  • Chimera Studio
• technique sft

  Supervised Fine-Tuning (SFT)

  Training technique where models learn from human-curated task-specific examples.

  Used for: Adapting models to specific tasks via labeled data

  • Zenfinity
  • Askimo
• technique lora

  LoRA / PEFT (Parameter-Efficient Fine-Tuning)

  Low-rank decomposition technique enabling fine-tuning with minimal parameter overhead.

  Used for: Memory-efficient adaptation of large models

  • Zenfinity
  • Beastfluence
• technique rlhf

  RLHF (Reinforcement Learning from Human Feedback)

  Training methodology using human feedback signals to guide model behavior.

  Used for: Aligning model outputs with human preferences

  • Zenfinity
• technique quantization

  Quantization

  Compression technique converting model weights to lower precision (int8, int4) while maintaining performance.

  Used for: Reducing model size and inference latency

  • Zenfinity
  • Askimo
  • Educational Voice Assistant
• technique vad

  Voice Activity Detection (VAD)

  ML-based audio filtering to detect presence/absence of human speech.

  Used for: Real-time speech detection for voice agents

  • Askimo
  • VocalEats
  • ATC Communication
• technique semantic-search

  Semantic Search

  Information retrieval using dense embeddings to find meaning-level similarity.

  Used for: Finding semantically similar content from text embeddings

  • Educational Voice Assistant
  • RAG System for Clinics
  • MarketingHub
• technique tool-calling

  Tool Calling / Function Calling

  Technique enabling models to invoke external tools and APIs as part of reasoning.

  Used for: Extending LLMs with external API integration

  • Askimo
  • Memecoin Agent
  • Solana DeFi Framework
  • MarketingHub
• technique inpainting

  Inpainting

  Generative technique for filling masked regions with coherent, context-aware content.

  Used for: Context-aware image completion and editing

  • Chimera Studio
• model yl4579/StyleTTS2

  StyleTTS 2

  Reference implementation and documentation for StyleTTS 2 — style diffusion plus adversarial training with a speech language model.

  Used for: Text-to-speech synthesis; fully fine-tuned onto a Hebrew single-speaker corpus

  • Hebrew TTS
• paper 2306.07691

  StyleTTS 2: Towards Human-Level Text-to-Speech

  Introduces style diffusion and adversarial training with large speech language models for human-level TTS.

  Used for: The method behind the fine-tune: style diffusion and SLM adversarial training

  • Hebrew TTS
• model yl4579/PL-BERT

  PL-BERT

  Phoneme-level BERT for text-to-speech, used as the text encoder in the StyleTTS 2 stack.

  Used for: Phoneme-level text encoding — what makes a non-English target tractable

  • Hebrew TTS
• paper 2010.05646

  HiFi-GAN

  Generative adversarial network for efficient, high-fidelity speech synthesis; the decoder in the StyleTTS 2 configuration.

  Used for: Waveform decoding from mel-spectrograms

  • Hebrew TTS
• model yl4579/StyleTTS

  StyleTTS (original)

  The first StyleTTS. Introduced the style encoder and AdaIN-conditioned decoder that StyleTTS 2 keeps; reading it first is what makes the v2 config legible.

  Used for: Lineage of the architecture that was fine-tuned

  • Hebrew TTS
• paper arXiv:2205.15439

  StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis

  Li, Han and Mesgarani, 2022. Establishes style as a sampled vector rather than a speaker embedding — the idea the diffusion sampler in v2 replaces.

  Used for: Where the style vector and AdaIN conditioning come from

  • Hebrew TTS
• paper arXiv:1904.02882

  LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech

  Zen et al., 2019. 585 hours at 24 kHz. The fine-tune inherits its sample rate, its 24 kHz mel configuration and its prosody prior from a checkpoint trained on this.

  Used for: The corpus behind the pretrained checkpoint the fine-tune starts from

  • Hebrew TTS
• model openslr/60

  LibriTTS corpus (OpenSLR 60)

  The distributed corpus. Its phonemised transcripts are the 141,433-line OOD text pool the SLM adversarial stage samples from.

  Used for: Source of the out-of-distribution text set used in adversarial training

  • Hebrew TTS
• technique nltk

  NLTK

  The punkt tokenizer splits a long script into sentences so each is synthesised as its own utterance. Getting this boundary wrong is what makes long-form generation drift.

  Used for: Sentence segmentation at inference, before phonemisation

  • Hebrew TTS
• technique espeak-ng

  espeak-ng

  The G2P backend. Every training transcript in this project is espeak-ng en-us IPA, which is also the source of the loanword problem: it has no entry for transliterated Hebrew or Aramaic.

  Used for: Grapheme-to-phoneme conversion into IPA

  • Hebrew TTS
• technique phonemizer

  phonemizer

  The Python wrapper that produced the phoneme strings in train_list.txt and val_list.txt, with stress marks preserved.

  Used for: Driving espeak-ng over the corpus and over inference text

  • Hebrew TTS
• technique huggingface/transformers

  Transformers (Hugging Face)

  A hard dependency of the training loop, not an optional convenience — the SLM discriminator is a Transformers checkpoint loaded at run time.

  Used for: Loading the speech language model used for adversarial supervision

  • Hebrew TTS
  • Askimo
  • Zenfinity
  • Medical RAG
• model microsoft/wavlm-base-plus

  WavLM

  Named directly in the training config: 13 layers, 768 hidden, resampled to 16 kHz. Its representations are what the generator is pushed to be indistinguishable under.

  Used for: The speech language model behind the adversarial loss

  • Hebrew TTS
• paper arXiv:2110.13900

  WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing

  Chen et al., 2021. Explains the layer-wise representations that make WavLM a better naturalness critic than a raw waveform discriminator.

  Used for: Why an SSL speech model makes a usable discriminator

  • Hebrew TTS
• paper arXiv:2005.11129

  Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

  Kim et al., 2020. Source of monotonic alignment search. The lambda_mono term in the config is this idea; the monotonic_align package is a literal dependency.

  Used for: The monotonic alignment objective scheduled in at TMA_epoch

  • Hebrew TTS
• paper arXiv:2006.04558

  FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

  Ren et al., 2020. The variance-adaptor argument behind separate duration and F0 losses — which is why rhythm and pauses can be reasoned about at all here.

  Used for: Why duration, pitch and energy are predicted as explicit targets

  • Hebrew TTS
• paper arXiv:2106.06103

  Conditional VAE with Adversarial Learning for End-to-End TTS (VITS)

  Kim, Kong and Son, 2021. The main end-to-end rival to StyleTTS 2; rejected here because no pretrained checkpoint on the target voice's acoustic conditions was available.

  Used for: The alternative architecture considered and not taken

  • Hebrew TTS
• paper arXiv:2301.02111

  Neural Codec Language Models are Zero-Shot TTS Synthesizers (VALL-E)

  Wang et al., 2023. Clones a voice from a 3-second prompt and carries emotion and acoustic environment with it. Attractive until you need one identity to hold for eleven minutes.

  Used for: The one-shot alternative to fine-tuning, and why it was not used

  • Hebrew TTS
• paper arXiv:2304.09116

  NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech Synthesizers

  Shen et al., 2023. The closest published relative of the style-diffusion module used here, extended to zero-shot speaker and singing transfer.

  Used for: Diffusion for prosody and one-shot identity transfer

  • Hebrew TTS
• paper arXiv:2112.02418

  YourTTS: Zero-Shot Multi-Speaker TTS and Voice Conversion

  Casanova et al., 2021. The obvious cheap route for a single-speaker target; documented speaker-similarity limits are why a full fine-tune was chosen instead.

  Used for: Zero-shot speaker adaptation as the baseline to beat

  • Hebrew TTS
• paper arXiv:2204.02152

  UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022

  Saeki et al., 2022. Predicts a mean opinion score without a listening panel — the standard way to compare TTS checkpoints when validation loss has stopped correlating with quality.

  Used for: Automatic MOS prediction as a check on subjective listening

  • Hebrew TTS
• technique NISQA

  NISQA

  Deep-learning MOS estimation with per-dimension output — noisiness, coloration, discontinuity, loudness. Discontinuity is the dimension that catches long-form seam artefacts.

  Used for: No-reference speech quality scoring of generated audio

  • Hebrew TTS
• technique runpod

  RunPod

  Per-hour GPU with a JupyterLab endpoint. The 100-epoch second stage ran here in 6 h 09 m; renting beats owning when a project needs one GPU for one day.

  Used for: Rented GPU for the fine-tuning run

  • Hebrew TTS
• technique wandb

  Weights & Biases

  Run comparison, hyperparameter history and artefact versioning — the record of which config produced which checkpoint, which is the question you cannot answer later from the filesystem alone.

  Used for: Experiment tracking across training runs

  • Hebrew TTS
  • Zenfinity
  • Chimera Studio
• technique tensorboard

  TensorBoard

  Written throughout the run. The 118 MB event file is almost entirely synthesised audio logged at each validation pass — for TTS the previews matter more than the scalars.

  Used for: Per-epoch loss curves and logged audio previews

  • Hebrew TTS
• technique huggingface-hub

  Hugging Face Hub

  Where the StyleTTS 2 LibriTTS checkpoint and microsoft/wavlm-base-plus are pulled from at setup time.

  Used for: Sourcing pretrained checkpoints and the SLM discriminator

  • Hebrew TTS
  • Askimo
  • Medical RAG
  • Zenfinity
• technique wada-snr

  WADA-SNR

  Kim & Stern, Interspeech 2008. Estimates SNR from the waveform amplitude distribution with no clean reference — which is the only way to gate a corpus you did not record yourself.

  Used for: Blind signal-to-noise scoring of every candidate training clip

  • Hebrew TTS

No references match that filter.