aims:

• methodo
• how ppl do llm training for hobbyist purpose

what i can use

• explain how I dev my side projects
• idea: schizodev must masking bluearchive char name in training phase
  note and what cool
• old ver. llama 3.1 from 2024
• huggingface disabled lol
• focus on translate so this time use translategamma-4b-it
  • still big model (support 60 langs while we need 1 )
• architecture
  • prompt format example
• get data
  • dataset scraped from reddit (~7k comments)
  • EDA:
    • rank of most used emoji
    • most mentioned characters
    • most upvoted comment
  • clean
    • rule based
      • handle
        • duplicates
          • flag duplicates
        • image only comments
          • flag if no text
    • scoring system
      • how it works
        • S_deg = degeneracy density, 0 to 1
          • goal: reward comments dense with the target dialect (emoji + slang (from EDA))
        • S_coh = structural coherency, 0 to 1
          • goal: penalise nonsense and unreadable wall of text entries
        • P_noise
          • comments that include URLs, usernames, … (avoid hallu urls)
      • handle
        • low quality
        • normal post
  • data pipeline funnel
  • formal input text - synthetic data
    • deepseek v4
  • training
    • gtx 1080
    • fine-tuninig llm
    • model 4B so it uses QLoRA (4-bit) VRAM around 3.5GB
    • 16 hrs training time (v1-6 with diff score cutoff)
  • CPU inferences
  • vibe code translating website
  • speculate GPU usage when running translate
  • unsloth does not works idk why

created on: Sun Jun 28 2026