⚓ T391311 ContentTranslation: DBQueryError: Error 1062: Duplicate entry for key 'cx_corpora_unique' when saving
Page Menu
Phabricator
Create Task
Maniphest
T391311
ContentTranslation: DBQueryError: Error 1062: Duplicate entry for key 'cx_corpora_unique' when saving
Closed, Resolved
Public
4 Estimated Story Points
PRODUCTION ERROR
Actions
Edit Task
Edit Related Tasks...
Create Subtask
Edit Parent Tasks
Edit Subtasks
Merge Duplicates In
Close As Duplicate
Edit Related Objects...
Edit Commits
Edit Mocks
Mute Notifications
Protect as security issue
Assigned To
abi_
Authored By
abi_
Apr 8 2025, 4:37 AM
2025-04-08 04:37:58 (UTC+0)
Tags
Wikimedia-production-error
(Apr-Jun 2025)
ContentTranslation
(Drafts and publishing)
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
(Done)
MW-1.44-notes (1.44.0-wmf.28; 2025-05-06)
MW-1.45-notes (1.45.0-wmf.8; 2025-07-01)
Referenced Files
F62865185: image.png
Jul 4 2025, 10:36 AM
2025-07-04 10:36:12 (UTC+0)
F62431772: image.png
Jun 23 2025, 10:12 AM
2025-06-23 10:12:11 (UTC+0)
F62372887: image.png
Jun 18 2025, 8:43 AM
2025-06-18 08:43:49 (UTC+0)
Subscribers
abi_
Aklapper
ngkountas
Nikerabbit
Description
Error
mwversion: 1.44.0-wmf.23
reqId:
aab97bd7-d3e5-4604-9c3c-daff7cd919a5
Find reqId in Logstash
normalized_message
[{reqId}] {exception_url} Wikimedia\Rdbms\DBQueryError: Error 1062: Duplicate entry '3012623-user-69886003_4_22-20250408011409' for key 'cx_corpora_unique'
Function: ContentTranslation\Store\TranslationCorporaStore::insertTranslationUnit
Query: INSERT INTO `cx_corpora` (cxc_translation_id,cxc_section_id,cxc_origin,cxc_sequence_id,cxc_timestamp,cxc_content) VALUES (3012623,'69886003_4_22','user',NULL,'20250408011409','
Frame
Location
Call
from
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/Database.php(1230)
#0
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/Database.php(1214)
Wikimedia\Rdbms\Database->getQueryException(string, int, string, string)
#1
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/Database.php(1188)
Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, int, string, string)
#2
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/Database.php(647)
Wikimedia\Rdbms\Database->reportQueryError(string, int, string, string, bool)
#3
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/Database.php(1496)
Wikimedia\Rdbms\Database->query(Wikimedia\Rdbms\Query, string)
#4
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/DBConnRef.php(127)
Wikimedia\Rdbms\Database->insert(string, array, string, array)
#5
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/database/DBConnRef.php(407)
Wikimedia\Rdbms\DBConnRef->__call(string, array)
#6
/srv/mediawiki/php-1.44.0-wmf.23/includes/libs/rdbms/querybuilder/InsertQueryBuilder.php(340)
Wikimedia\Rdbms\DBConnRef->insert(string, array, string, array)
#7
/srv/mediawiki/php-1.44.0-wmf.23/extensions/ContentTranslation/includes/Store/TranslationCorporaStore.php(99)
Wikimedia\Rdbms\InsertQueryBuilder->execute()
#8
/srv/mediawiki/php-1.44.0-wmf.23/extensions/ContentTranslation/includes/Store/TranslationCorporaStore.php(290)
ContentTranslation\Store\TranslationCorporaStore->insertTranslationUnit(ContentTranslation\Entity\TranslationUnit)
#9
/srv/mediawiki/php-1.44.0-wmf.23/extensions/ContentTranslation/includes/Manager/TranslationCorporaManager.php(80)
ContentTranslation\Store\TranslationCorporaStore->save(ContentTranslation\Entity\TranslationUnit, bool)
#10
/srv/mediawiki/php-1.44.0-wmf.23/extensions/ContentTranslation/includes/ActionApi/ApiContentTranslationSave.php(109)
ContentTranslation\Manager\TranslationCorporaManager->saveTranslationUnits(ContentTranslation\Translation, string)
#11
/srv/mediawiki/php-1.44.0-wmf.23/includes/api/ApiMain.php(2005)
ContentTranslation\ActionApi\ApiContentTranslationSave->execute()
#12
/srv/mediawiki/php-1.44.0-wmf.23/includes/api/ApiMain.php(947)
MediaWiki\Api\ApiMain->executeAction()
#13
/srv/mediawiki/php-1.44.0-wmf.23/includes/api/ApiMain.php(918)
MediaWiki\Api\ApiMain->executeActionWithErrorHandling()
#14
/srv/mediawiki/php-1.44.0-wmf.23/includes/api/ApiEntryPoint.php(152)
MediaWiki\Api\ApiMain->execute()
#15
/srv/mediawiki/php-1.44.0-wmf.23/includes/MediaWikiEntryPoint.php(202)
MediaWiki\Api\ApiEntryPoint->execute()
#16
/srv/mediawiki/php-1.44.0-wmf.23/api.php(44)
MediaWiki\MediaWikiEntryPoint->run()
#17
/srv/mediawiki/w/api.php(3)
require(string)
#18
{main}
Impact
Unsure. Verify that this doesn't block saving of translation and the user is displayed a proper message.
Notes
More issues:
Details
Request URL
Related Changes in Gerrit:
Subject
Repo
Branch
Lines +/-
TranslationCorporaStore: Remove logging on duplicate translation unit
mediawiki/extensions/ContentTranslation
master
+3
-21
TranslationCorporaStore: Mark log on insertion as debug
mediawiki/extensions/ContentTranslation
master
+4
-1
TranslationCorporaManager: Remove code to detect duplicates
mediawiki/extensions/ContentTranslation
master
+0
-33
TranslationCorporaStore: Improve log when inserting translation unit
mediawiki/extensions/ContentTranslation
master
+2
-1
Translation unit: Add logging for when the section id is duplicated
mediawiki/extensions/ContentTranslation
master
+33
-0
TranslationCorporaStore: Skip insertion if unique key conflicts
mediawiki/extensions/ContentTranslation
master
+43
-5
Add channel for ContentTranslation logging
operations/mediawiki-config
master
+1
-0
Customize query in gerrit
Related Objects
Mentions
Mentioned In
T397910: Multiple simultaneous requests to "cxsave" action
Mentioned Here
T397910: Multiple simultaneous requests to "cxsave" action
Event Timeline
abi_
created this task.
Apr 8 2025, 4:37 AM
2025-04-08 04:37:58 (UTC+0)
Restricted Application
added a subscriber:
Aklapper
View Herald Transcript
Apr 8 2025, 4:37 AM
2025-04-08 04:37:59 (UTC+0)
Reedy
moved this task from
Untriaged
to
Apr-Jun 2025
on the
Wikimedia-production-error
board.
Apr 14 2025, 5:40 PM
2025-04-14 17:40:25 (UTC+0)
Nikerabbit
added a project:
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
Apr 15 2025, 10:47 AM
2025-04-15 10:47:37 (UTC+0)
Nikerabbit
triaged this task as
Medium
priority.
Apr 15 2025, 10:51 AM
2025-04-15 10:51:56 (UTC+0)
Nikerabbit
moved this task from
Backlog
to
Ready for dev
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
Nikerabbit
set the point value for this task to
abi_
changed the task status from
Open
to
In Progress
Apr 16 2025, 10:48 AM
2025-04-16 10:48:19 (UTC+0)
abi_
claimed this task.
abi_
moved this task from
Ready for dev
to
In Progress
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
gerritbot
added a comment.
Apr 16 2025, 10:49 AM
2025-04-16 10:49:08 (UTC+0)
Comment Actions
Change #1136975 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[operations/mediawiki-config@master] Add channel for ContentTranslation logging
gerritbot
added a project:
Patch-For-Review
Apr 16 2025, 10:49 AM
2025-04-16 10:49:09 (UTC+0)
gerritbot
added a comment.
Apr 16 2025, 10:54 AM
2025-04-16 10:54:49 (UTC+0)
Comment Actions
Change #1136977 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Skip insertion if unique key conflicts
gerritbot
added a comment.
Apr 23 2025, 7:06 AM
2025-04-23 07:06:53 (UTC+0)
Comment Actions
Change #1136975
merged
by jenkins-bot:
[operations/mediawiki-config@master] Add channel for ContentTranslation logging
Stashbot
added a comment.
Apr 23 2025, 7:07 AM
2025-04-23 07:07:33 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2025-04-23T07:07:32Z]
T391311
)]]
Stashbot
added a comment.
Apr 23 2025, 7:12 AM
2025-04-23 07:12:23 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2025-04-23T07:12:23Z]
T391311
)]] synced to the testservers (
Stashbot
added a comment.
Apr 23 2025, 7:24 AM
2025-04-23 07:24:27 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2025-04-23T07:24:26Z]
T391311
)]] (duration: 16m 53s)
abi_
added a comment.
Apr 28 2025, 5:04 AM
2025-04-28 05:04:25 (UTC+0)
Comment Actions
I've submitted a
patch
to ignore insertion on unique key constraint failures. In addition, we've added some logging to check if there is data loss when these conflict occurs.
We need to know both the inserted and ignored value to determine if they are different and compare the requests to see if there is something else going on.
The additional debug logs would be temporary. We will remove them once we confirm whether there is data loss.
We could potentially just query the database directly in the failing request and compare the data. We would not get some other details like
reqId
to further inspect the requests (e.g. are they happening at the same time).
Related discussion on Gerrit:
gerritbot
added a comment.
Apr 30 2025, 10:29 AM
2025-04-30 10:29:06 (UTC+0)
Comment Actions
Change #1136977
merged
by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Skip insertion if unique key conflicts
Maintenance_bot
removed a project:
Patch-For-Review
Apr 30 2025, 10:31 AM
2025-04-30 10:31:24 (UTC+0)
ReleaseTaggerBot
added a project:
MW-1.44-notes (1.44.0-wmf.28; 2025-05-06)
Apr 30 2025, 11:00 AM
2025-04-30 11:00:20 (UTC+0)
Nikerabbit
moved this task from
In Progress
to
Pending Deployment
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
May 5 2025, 8:34 AM
2025-05-05 08:34:19 (UTC+0)
Nikerabbit
moved this task from
Needs Triage
to
Bugs
on the
ContentTranslation
board.
May 8 2025, 8:52 AM
2025-05-08 08:52:32 (UTC+0)
Nikerabbit
moved this task from
Pending Deployment
to
Need QA
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
May 12 2025, 8:14 AM
2025-05-12 08:14:10 (UTC+0)
abi_
added a comment.
May 12 2025, 12:19 PM
2025-05-12 12:19:23 (UTC+0)
Comment Actions
No longer seeing any occurrences of this error since May 8th:
The ContentTranslation channel is also active.
Not seeing any instances of:
'No translation unit inserted due to unique key conflict.'
. (ref:
I'm keeping an eye out to see if we get that error.
Nikerabbit
subscribed.
May 12 2025, 2:27 PM
2025-05-12 14:27:08 (UTC+0)
Comment Actions
In
T391311#10811175
@abi_
wrote:
Not seeing any instances of:
'No translation unit inserted due to unique key conflict.'
. (ref:
Search for e.g.
reqId:"17f9026e-66c4-49ab-a573-5b2d4548a964"
. Surprisingly, it looks like the duplicates are coming not coming from multiple requests, but within a single request.
abi_
added a comment.
May 20 2025, 11:28 AM
2025-05-20 11:28:43 (UTC+0)
Comment Actions
In
T391311#10811875
@Nikerabbit
wrote:
In
T391311#10811175
@abi_
wrote:
Not seeing any instances of:
'No translation unit inserted due to unique key conflict.'
. (ref:
Search for e.g.
reqId:"17f9026e-66c4-49ab-a573-5b2d4548a964"
. Surprisingly, it looks like the duplicates are coming not coming from multiple requests, but within a single request.
Thanks. I'm noticing that most of the errors are in the same request and from the desktop editor (based on the lack of
m.
in the URL)
Example request Id:
29585b30-1ee2-42c7-846e-eff4e56b6e88
bf4da81f-fca2-44f7-93d4-474da4d68bb7
17f9026e-66c4-49ab-a573-5b2d4548a964
My initial assumption is that the request from the desktop editor has duplicated translation units that are causing this error.
In
TranslationCorporaManager::createTranslationUnitsFromContent
, we create an array of
TranslationUnit
$translationUnits[] = new TranslationUnit(
(string)$translationUnitData['sectionId'],
$translationUnitData['origin'],
$sequenceId,
(string)$translationUnitData['content'], // Content can be null in case translator clear the section.
$translationId,
$timestamp,
$validate
);
Maybe we can create an array with the key being the
sectionId
so that we don't have this duplication.
The other possibility is that the
sectionId
is being duplicated somehow when adaptation happens on CXServer.
Needs more investigation.
Nikerabbit
moved this task from
Need QA
to
In Progress
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
May 20 2025, 11:39 AM
2025-05-20 11:39:12 (UTC+0)
gerritbot
added a comment.
May 21 2025, 5:10 PM
2025-05-21 17:10:15 (UTC+0)
Comment Actions
Change #1148919 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/ContentTranslation@master] Translation unit: Add logging for when the section id is duplicated
gerritbot
added a project:
Patch-For-Review
May 21 2025, 5:10 PM
2025-05-21 17:10:16 (UTC+0)
Nikerabbit
moved this task from
Bugs
to
Drafts and publishing
on the
ContentTranslation
board.
May 22 2025, 3:45 PM
2025-05-22 15:45:20 (UTC+0)
gerritbot
added a comment.
Jun 2 2025, 1:03 PM
2025-06-02 13:03:45 (UTC+0)
Comment Actions
Change #1148919
merged
by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Translation unit: Add logging for when the section id is duplicated
Maintenance_bot
removed a project:
Patch-For-Review
Jun 2 2025, 1:31 PM
2025-06-02 13:31:15 (UTC+0)
ReleaseTaggerBot
added a project:
MW-1.45-notes (1.45.0-wmf.4; 2025-06-03)
Jun 2 2025, 2:00 PM
2025-06-02 14:00:31 (UTC+0)
abi_
moved this task from
In Progress
to
Pending Deployment
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
Jun 3 2025, 8:02 AM
2025-06-03 08:02:23 (UTC+0)
abi_
added a comment.
Jun 18 2025, 8:43 AM
2025-06-18 08:43:49 (UTC+0)
Comment Actions
Looking at logs from the last 2 weeks:
, there are 215 occurrences of
No translation unit inserted due to unique key conflict.
but haven't seen any logs with the message
Duplicate translation unit found in the list of translation units to save.
The unique key conflict has occurred in the same request for the following request id:
3027b490-11d3-48fe-9a26-2534ba6b4f03
a8b0435d-3bc4-42aa-bf6e-c916afc3671b
5e88638e-5e61-4116-9cf8-85d027005346
and others.
Looking specifically
at the
reqId: 5e88638e-5e61-4116-9cf8-85d027005346
Every occurrence of
No translation unit inserted due to unique key conflict.
is preceded by
Inserting translation unit
of that same specific translation unit but not sure how that's happening since we don't have any logs with
Duplicate translation unit found in the list of translation units to save.
Nikerabbit
moved this task from
Pending Deployment
to
In Progress
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
Jun 23 2025, 8:33 AM
2025-06-23 08:33:36 (UTC+0)
ngkountas
subscribed.
Jun 23 2025, 10:12 AM
2025-06-23 10:12:11 (UTC+0)
Comment Actions
From my investigation, I came to the conclusion that the
No translation unit inserted due to unique key conflict
issue, occurs when there are two "cxsave" requests for the same translation units, handled by two different hosts. When this happens, the second request leads to this error. Below I attach a screenshot of the same requests listed in Abijeet's
comment
above, along with some markers that indicate the conflicting requests and the related error. Please note, that for some reason, the screenshot that
@abi_
posted above only lists requests from the same host.
What we really need to investigate is under which conditions, there are two "cxsave" requests for the same translation units, because obviously this is not something that happens all the time.
The screenshot was taken from this
Logstash view
gerritbot
added a comment.
Jun 23 2025, 10:55 AM
2025-06-23 10:55:06 (UTC+0)
Comment Actions
Change #1162864 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Improve log when inserting translation unit
gerritbot
added a project:
Patch-For-Review
Jun 23 2025, 10:55 AM
2025-06-23 10:55:07 (UTC+0)
abi_
added a comment.
Jun 23 2025, 11:30 AM
2025-06-23 11:30:24 (UTC+0)
Comment Actions
My understanding of the logs previously was incorrect. The "Inserting translation unit" log will appear every time irrespective of whether or not the translation unit was inserted. I've
updated the logging
to avoid this confusion.
With Nik's assesment and with the above knowledge, if we look at specific section ids and review the logs, its clear that the duplicate translation unit issue occurs across request Id, and not the same request Id.
See the following section Id as example:
1192508726_0_1
1196113243_0_0
1240542910_4_39
This makes a lot more sense.
gerritbot
added a comment.
Jun 23 2025, 11:39 AM
2025-06-23 11:39:14 (UTC+0)
Comment Actions
Change #1162870 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaManager: Remove code to detect duplicates
gerritbot
added a comment.
Jun 23 2025, 11:43 AM
2025-06-23 11:43:07 (UTC+0)
Comment Actions
Change #1162864
merged
by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Improve log when inserting translation unit
gerritbot
added a comment.
Jun 23 2025, 12:08 PM
2025-06-23 12:08:15 (UTC+0)
Comment Actions
Change #1162870
merged
by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaManager: Remove code to detect duplicates
Maintenance_bot
removed a project:
Patch-For-Review
Jun 23 2025, 12:31 PM
2025-06-23 12:31:36 (UTC+0)
gerritbot
added a comment.
Jun 23 2025, 12:46 PM
2025-06-23 12:46:43 (UTC+0)
Comment Actions
Change #1162896 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Remove logging on duplicate translation unit
gerritbot
added a project:
Patch-For-Review
Jun 23 2025, 12:46 PM
2025-06-23 12:46:43 (UTC+0)
ReleaseTaggerBot
edited projects, added
MW-1.45-notes (1.45.0-wmf.8; 2025-07-01)
; removed
MW-1.45-notes (1.45.0-wmf.4; 2025-06-03)
Jun 24 2025, 3:02 PM
2025-06-24 15:02:31 (UTC+0)
abi_
added a comment.
Jun 26 2025, 9:32 AM
2025-06-26 09:32:34 (UTC+0)
Comment Actions
I checked the logs some more and I can confirm my initial hypothesis about the message
No translation unit inserted due to unique key conflict.
only appearing across different requests. I'm going to remove the logs, and file a separate lower priority task to understand why we have multiple requests being sent to CX.
abi_
mentioned this in
T397910: Multiple simultaneous requests to "cxsave" action
Jun 26 2025, 9:53 AM
2025-06-26 09:53:19 (UTC+0)
Comment Actions
In
T391311#10949606
@abi_
wrote:
I checked the logs some more and I can confirm my initial hypothesis about the message
No translation unit inserted due to unique key conflict.
only appearing across different requests. I'm going to remove the logs, and file a separate lower priority task to understand why we have multiple requests being sent to CX.
Created:
T397910: Multiple simultaneous requests to "cxsave" action
gerritbot
added a comment.
Jun 26 2025, 10:32 AM
2025-06-26 10:32:39 (UTC+0)
Comment Actions
Change #1164142 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Mark log on insertion as debug
gerritbot
added a comment.
Jun 27 2025, 8:37 AM
2025-06-27 08:37:38 (UTC+0)
Comment Actions
Change #1164142
merged
by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Mark log on insertion as debug
gerritbot
added a comment.
Jun 29 2025, 1:34 AM
2025-06-29 01:34:51 (UTC+0)
Comment Actions
Change #1162896
abandoned
by Abijeet Patro:
[mediawiki/extensions/ContentTranslation@master] TranslationCorporaStore: Remove logging on duplicate translation unit
Reason:
In favor of I0262447f2acc7f8a4c8bd6349342a3c8fa87c4a5
abi_
moved this task from
In Progress
to
Pending Deployment
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
Jun 29 2025, 1:35 AM
2025-06-29 01:35:04 (UTC+0)
Maintenance_bot
removed a project:
Patch-For-Review
Jun 29 2025, 2:30 AM
2025-06-29 02:30:20 (UTC+0)
Nikerabbit
moved this task from
Pending Deployment
to
Need QA
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
Jul 3 2025, 11:54 AM
2025-07-03 11:54:32 (UTC+0)
abi_
closed this task as
Resolved
Jul 4 2025, 10:36 AM
2025-07-04 10:36:12 (UTC+0)
abi_
moved this task from
Need QA
to
Done
on the
LPL Essential (LPL Essential 2025 Apr-Jun: CX)
board.
Comment Actions
I'm not seeing any more logs about inserting translation units
Summary of the problem:
Log In to Comment
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct.
Wikimedia Foundation
Code of Conduct
Disclaimer
CC-BY-SA
GPL
Credits
US