Skip to contentSkip to main navigation Skip to footer

Language Identification – Languages

Recognized languages

Languages pre-trained in the default language pack are listed in the table below, each LID generation is a separate column (in the 4th generation we switched to using language tags instead of names):

L4

L3, XL3S2, L2
sq-AL (Albanian)AlbanianAlbanian
am-ET (Amharic)AmharicAmharic
ar-EG (Arabic_Egypt)Arabic
ar-KW (Arabic_Gulf)Arabic_Gulf
ar-IQ (Arabic_Iraqi)Arabic_IraqiArabic_Iraqi
ar-XL (Arabic_Levantine)Arabic_LevantineArabic_Levantine
ar-MA (Arabic_Maghrebi)Arabic_Maghrebi
arb (Arabic_MSA)Arabic_MSAArabic_MSA
as-IN (Assamese)
az-AZ (Azerbaijani)AzerbaijaniAzerbaijani
bn-BD (Bangla_Bengali)Bangla_BengaliBangla_Bengali
be-BY (Belarusian)
bg-BG (Bulgarian)
my-MM (Burmese)BurmeseBurmese
ceb-PH (Cebuano)
zh-HK (Chinese_Cantonese)Chinese_CantoneseChinese_Cantonese
zh-CN (Chinese_Mandarin)Chinese_MandarinChinese_Mandarin
nan-CN (Chinese_Min_Nan)Chinese_Dialects
wuu-CN (Chinese_Wu)Chinese_Dialects
cv-RU (Chuvash)
CreoleCreole
cs-CZ (Czech)CzechCzech
fa-AF (Dari)DariDari
nl (Dutch)
en-US (English_American)English_AmericanEnglish_American
en-GB (English_British)English_BritishEnglish_British
en-IN (English_Indian)English_Indian
fa-IR (Farsi)FarsiFarsi
fr (French)FrenchFrench
ka-GE (Georgian)GeorgianGeorgian
de (German)GermanGerman
el-GR (Greek)GreekGreek
gn (Guarani)
ht-HT (Haitian_Creole)
ha (Hausa)HausaHausa
HebrewHebrew
hi-IN (Hindi)HindiHindi
hu-HU (Hungarian)Hungarian
id-ID (Indonesian)IndonesianIndonesian
it (Italian)ItalianItalian
ja-JP (Japanese)JapaneseJapanese
kk-KZ (Kazakh)
km (Khmer)KhmerKhmer
rn-BI (Kirundi_Kinyarwanda)Kirundi_KinyarwandaKirundi_Kinyarwanda
ko-KR (Korean)KoreanKorean
ku (Kurdish)
lo-LA (Lao)LaoLao
lt-LT (Lithuanian)
lb-LU (Luxembourgish)
mk-MK (Macedonian)MacedonianMacedonian
nd-ZW (Ndebele)NdebeleNdebele
om (Oromo)Afan_OromoAfan_Oromo
ps (Pashto)PashtoPashto
pl-PL (Polish)PolishPolish
pt (Portuguese)PortuguesePortuguese
pa (Punjabi)Punjabi
ro-RO (Romanian)
ru-RU (Russian)RussianRussian
sh (Serbo-Croat-Bosnian)SerbianSerbian
sh (Serbo-Croat-Bosnian)CroatianCroatian
sh (Serbo-Croat-Bosnian)BosnianBosnian
sn (Shona)ShonaShona
sk-SK (Slovak)SlovakSlovak
sl-SI (Slovenian)
so (Somali)SomaliSomali
es-XA (Spanish_American)
es-ES (Spanish_European)SpanishSpanish
sw (Swahili)SwahiliSwahili
sv-SE (Swedish)Swedish
tl-PH (Tagalog)Tagalog
ta (Tamil)TamilTamil
te-IN (Telugu)
th-TH (Thai)ThaiThai
bo (Tibetan)TibetanTibetan
ti (Tigrignya)TigrignaTigrigna
tpi-PG (Tok_Pisin)
tr-TR (Turkish)TurkishTurkish
uk-UA (Ukrainian)UkrainianUkrainian
ur (Urdu)UrduUrdu
uz-UZ (Uzbek)UzbekUzbek
vi-VN (Vietnamese)VietnameseVietnamese
zu (Zulu)