🔍 Read the full analysis: How Egyptian And Nigerian English Joined The Open ASR Leaderboard on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face has added two private Liva AI evaluation splits to its Open ASR Leaderboard, covering Egyptian- and Nigerian-accented English. The conversational recordings total 2.89 hours; the announcement does not establish how representative they are of speakers in either country or report their effect on leaderboard scores.
Hugging Face has added two private English speech evaluation splits from Liva AI to its Open ASR Leaderboard, covering Egyptian and Nigerian accents, as described in the original analysis. The recordings total 2.89 hours and add two African English varieties to the leaderboard’s accent comparisons, though the announcement does not report how the new data affects model scores.
The new sets are named Liva AI Conversational EG and Liva AI Conversational NG. The Egyptian split contains 1.34 hours of speech, and the Nigerian split contains 1.55 hours. Hugging Face describes both as spontaneous conversational audio. Their transcripts are punctuated and cased, and include disfluencies, such as hesitations and repetitions.
The splits are private evaluation data, rather than datasets available to download and use as ordinary public benchmarks. Hugging Face says private evaluation material is intended to make assessment more trustworthy and reduce developers’ ability to tune systems against publicly available test examples. The announcement does not provide score comparisons showing whether the new splits change model rankings or reveal differences in system performance.
Hugging Face also says the leaderboard’s Private tab will present two accent comparisons: US versus non-US English, and English grouped as Inner Circle versus Outer and Expanding Circles. The non-US group includes the UK, Australia, Canada, India, Nigeria and Egypt. The post credits Hume AI with the idea for these leaderboard groupings and says they draw on Kachru’s Three Circles framework.
Egyptian and Nigerian English join the Open ASR Leaderboard
Hugging Face added private Liva AI speech splits to broaden accent evaluation. The short recordings add coverage, while leaving questions about how representative the results are.
What was added
Both datasets contain spontaneous conversation with punctuated, cased transcripts that retain disfluencies.
Liva AI Conversational EG
A private evaluation split for Egyptian-accented English.
Liva AI Conversational NG
A private evaluation split for Nigerian-accented English.
New ways to compare accents
The leaderboard’s Private tab will show results across two groupings.
- Non-US includes the UK, Australia, Canada, India, Nigeria and Egypt.
- Inner Circle compared with Outer and Expanding Circles, based on Kachru’s framework.
- Hugging Face plans to separate the latter circles as more accents are added.
Why private evaluation matters
Keeping test examples private may make it harder to tune systems directly to known benchmark material.
Users can inspect recognition results for two additional English varieties.
Scores describe these specific short test sets, not all speakers in either country.
What the New Accent Tests Add
The addition gives leaderboard users a way to inspect speech-recognition evaluation that includes Egyptian and Nigerian English, rather than relying only on broader scores or the accents already represented. Recognition performance can vary across speech varieties, so an overall score may conceal differences in how well a system handles particular speakers. Accent-specific results can help developers and users ask more targeted questions about model performance.
But the new splits are small in duration, with less than three hours combined. Their inclusion broadens the leaderboard’s coverage, but does not by itself show that the recordings represent the range of English spoken in Egypt or Nigeria. Scores should be understood as results on these particular test sets, not as a definitive measure of recognition quality for either country.
Keeping test material private may make it harder to optimize a model for the exact examples used in evaluation. That can help preserve the usefulness of a benchmark, but privacy also means outside readers cannot directly inspect the test recordings or independently check the data’s breadth from the announcement. The practical value of the additions will depend partly on the detail available about the speakers, collection process and resulting scores.
Popular right now
Bose QuietComfort Headphones (2nd Gen), Wireless Over-Ear Noise-Cancelling Headphones, Black
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
Breville Bambino Espresso Machine, Brushed Stainless Steel
As an affiliate, we earn on qualifying purchases.
Earlier Private Accent Splits
The Liva AI additions continue the leaderboard’s use of private speech data. According to Hugging Face, it previously added 11 private splits from Appen and DataOceanAI, covering scripted and conversational speech from five accents. It later added a private split for Indian-accented English in collaboration with Voice Arena.
The new accent categories use Kachru’s Three Circles framework. In the current presentation described by Hugging Face, the Inner Circle grouping includes the US, UK, Australia and Canada, while India, Nigeria and Egypt are grouped under Outer and Expanding Circles. The post says the leaderboard plans to separate Outer and Expanding Circles when more accents are available, but does not specify when that will happen.
“We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian and Nigerian.”
— Hugging Face
What the Split Details Do Not Show
The announcement does not say when the recordings were collected, how many speakers took part, or how those speakers were selected. It provides no breakdown by region, age or other speaker characteristics. Without those details, readers cannot assess how well a short set of conversations reflects variation among Egyptian and Nigerian English speakers.
Hugging Face also does not provide baseline scores or ranking changes associated with the new splits. The post describes the data as private but does not say whether recordings or fuller dataset documentation could become accessible later. It remains unclear what minimum number or range of accents will prompt the planned separation of Outer and Expanding Circles.
More Accents and Separate Groups
Hugging Face says it plans to continue adding accents and languages to the leaderboard. It also plans to split the Outer and Expanding Circles into separate groupings as more accents become available. No timeline or numerical threshold was announced for that change.
For now, users can consult the leaderboard’s Private tab for the stated comparisons. Further updates could clarify speaker composition, collection methods and per-accent results, and show how the new test sets affect model evaluations. The announcement does not identify a date for those details or for additional data releases.
Key Questions
Which accents were added to the Open ASR Leaderboard?
The new private splits evaluate Egyptian-accented and Nigerian-accented English through Liva AI Conversational EG and Liva AI Conversational NG.
How much speech do the two splits contain?
The Egyptian split contains 1.34 hours, and the Nigerian split contains 1.55 hours, for a combined total of 2.89 hours.
What kind of recordings are included?
Hugging Face describes the audio as spontaneous conversation. The transcripts are punctuated and cased and include disfluencies.
Can the public download the evaluation data?
No. The announcement describes the splits as private evaluation data, not publicly available benchmark datasets.
Do the additions show how well ASR systems perform across Egypt and Nigeria?
They provide evaluation results on specific short test sets, but the announcement gives no speaker or collection details to establish how representative the data is of either country’s English speakers.
Primary source: Hugging Face · via ThorstenMeyerAI.com
