⚓ T335972 Launch content separation (Zebra #9) A/B test
Page Menu
Phabricator
Create Task
Maniphest
T335972
Launch content separation (Zebra #9) A/B test
Closed, Resolved
Public
2 Estimated Story Points
Actions
Edit Task
Edit Related Tasks...
Create Subtask
Edit Parent Tasks
Edit Subtasks
Merge Duplicates In
Close As Duplicate
Edit Related Objects...
Edit Commits
Edit Mocks
Mute Notifications
Protect as security issue
Assigned To
jwang
Authored By
ovasileva
May 4 2023, 3:16 PM
2023-05-04 15:16:21 (UTC+0)
Tags
Vector 2022
(Incoming)
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 4)
(Ready for Signoff)
MW-1.41-notes (1.41.0-wmf.10; 2023-05-23)
Referenced Files
F37001843: Screenshot 2023-05-16 at 5.10.38 PM.png
May 16 2023, 10:52 PM
2023-05-16 22:52:57 (UTC+0)
F37001840: query-hive-32809.csv
May 16 2023, 10:52 PM
2023-05-16 22:52:57 (UTC+0)
F37001841: query-hive-32812.csv
May 16 2023, 10:52 PM
2023-05-16 22:52:57 (UTC+0)
Subscribers
Aklapper
Huji
Izno
Jdlrobson
jwang
KSarabia-WMF
Ladsgroup
View All 11 Subscribers
Description
Background
We are now ready to launch the a/b test for the content separation work
Acceptance criteria
Enable a/b test for the following wikis on
Wed May 17th
Monday 5/29: hewiki
Enable a/b test for the following wikis on
Thur May 18th
Tuesday 5/30: enwiki, fawiki, frwiki, kowiki, ptwiki, srwiki, thwiki, trwiki, viwiki
Signoff criteria
Ping
@jwang
after deployment for A/B test schema QA
Ensure A/B test is disabled
two weeks
after being initially enabled. The interface should go back to the control state (Edit: See follow up ticket:
Document on
Details
Related Changes in Gerrit:
Subject
Repo
Branch
Lines +/-
Make ABRequirement aware of Require
mediawiki/skins/Vector
master
+17
-4
Launch content separation Zebra AB Test
operations/mediawiki-config
master
+36
-0
Enable Vector "Zebra" AB test on Hebrew wikipedia
operations/mediawiki-config
master
+9
-2
Use document feature classes to extract A/B test state
mediawiki/skins/Vector
wmf/1.41.0-wmf.10
+5
-0
Use document feature classes to extract A/B test state
mediawiki/skins/Vector
master
+5
-0
Override mw.experiments.getBucket() via body classes
mediawiki/skins/Vector
master
+15
-1
Enable zebra ab test in hewiki
operations/mediawiki-config
master
+13
-2
[Beta] Enable Vector AB test on beta spanish wiki
operations/mediawiki-config
master
+19
-0
Enable Vector "Zebra" AB test on beta cluster.
operations/mediawiki-config
master
+11
-8
Customize query in gerrit
Related Objects
Search...
Task Graph
Mentions
Status
Subtype
Assigned
Task
Resolved
ovasileva
T333180
[EPIC] A/B test Zebra #9 prototype
Resolved
jwang
T335972
Launch content separation (Zebra #9) A/B test
Resolved
jwang
T336299
QA A/B test schema for Zebra #9 A/B test
Resolved
bwang
T336969
[Zebra AB test] Fix the mixing of global and user IDs for AB Test Enrollment Bucketing
Resolved
KSarabia-WMF
T337956
Turn off Zebra A/B Test
Resolved
Mabualruz
T339957
Zebra: Completely remove any A/B test-related code
Mentioned In
T327718: Increase space between groups in main menu and page tools menu
T333222: Create instrumentation spec for Zebra #9 prototype
T308134: Deploy "add a link" to 9th round of wikis
T317375: [Config change] Deploy New Topic Tool as opt-out preference at fi.wiki (desktop)
Mentioned Here
T337956: Turn off Zebra A/B Test
T308134: Deploy "add a link" to 9th round of wikis
T317375: [Config change] Deploy New Topic Tool as opt-out preference at fi.wiki (desktop)
Event Timeline
ovasileva
created this task.
May 4 2023, 3:16 PM
2023-05-04 15:16:21 (UTC+0)
Restricted Application
added subscribers:
Huji
revi
Aklapper
View Herald Transcript
May 4 2023, 3:16 PM
2023-05-04 15:16:21 (UTC+0)
ovasileva
triaged this task as
High
priority.
May 4 2023, 3:16 PM
2023-05-04 15:16:31 (UTC+0)
ovasileva
added a project:
Vector 2022
ovasileva
added a parent task:
T333180: [EPIC] A/B test Zebra #9 prototype
May 4 2023, 5:24 PM
2023-05-04 17:24:26 (UTC+0)
LGoto
set the point value for this task to
May 4 2023, 5:24 PM
2023-05-04 17:24:37 (UTC+0)
Ladsgroup
subscribed.
May 4 2023, 5:32 PM
2023-05-04 17:32:07 (UTC+0)
Izno
subscribed.
May 4 2023, 8:16 PM
2023-05-04 20:16:22 (UTC+0)
ovasileva
moved this task from
Incoming
to
Ready for Development
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 3)
board.
May 8 2023, 2:21 PM
2023-05-08 14:21:57 (UTC+0)
KSarabia-WMF
claimed this task.
May 8 2023, 4:58 PM
2023-05-08 16:58:54 (UTC+0)
Jdlrobson
updated the task description.
(Show Details)
May 8 2023, 5:24 PM
2023-05-08 17:24:00 (UTC+0)
KSarabia-WMF
moved this task from
Ready for Development
to
Doing
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 3)
board.
May 9 2023, 3:40 PM
2023-05-09 15:40:09 (UTC+0)
ovasileva
updated the task description.
(Show Details)
May 9 2023, 7:06 PM
2023-05-09 19:06:10 (UTC+0)
gerritbot
added a comment.
May 10 2023, 4:56 PM
2023-05-10 16:56:34 (UTC+0)
Comment Actions
Change 918568 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):
[operations/mediawiki-config@master] Launch content separation Zebra AB Test
gerritbot
added a project:
Patch-For-Review
May 10 2023, 4:56 PM
2023-05-10 16:56:34 (UTC+0)
gerritbot
added a comment.
May 10 2023, 6:20 PM
2023-05-10 18:20:32 (UTC+0)
Comment Actions
Change 918584 had a related patch set uploaded (by Jdrewniak; author: Jdrewniak):
[operations/mediawiki-config@master] Enable Vector "Zebra" AB test on beta cluster.
gerritbot
added a comment.
May 10 2023, 8:24 PM
2023-05-10 20:24:27 (UTC+0)
Comment Actions
Change 918584
merged
by jenkins-bot:
[operations/mediawiki-config@master] Enable Vector "Zebra" AB test on beta cluster.
Jdlrobson
moved this task from
Doing
to
Blocked on Others
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 3)
board.
May 10 2023, 8:36 PM
2023-05-10 20:36:16 (UTC+0)
Jdlrobson
updated the task description.
(Show Details)
gerritbot
added a comment.
May 11 2023, 12:48 AM
2023-05-11 00:48:52 (UTC+0)
Comment Actions
Change 918613 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):
[mediawiki/skins/Vector@master] Make ABRequirement aware of Require
Jdlrobson
moved this task from
Blocked on Others
to
Doing
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 3)
board.
May 16 2023, 4:44 PM
2023-05-16 16:44:41 (UTC+0)
gerritbot
added a comment.
May 16 2023, 10:41 PM
2023-05-16 22:41:03 (UTC+0)
Comment Actions
Change 920386 had a related patch set uploaded (by Jdlrobson; author: Kimberly Sarabia):
[operations/mediawiki-config@master] Enable zebra ab test in hewiki
Jdlrobson
moved this task from
Doing
to
Code Review
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 3)
board.
May 16 2023, 10:44 PM
2023-05-16 22:44:29 (UTC+0)
KSarabia-WMF
added a comment.
Edited
May 16 2023, 10:52 PM
2023-05-16 22:52:57 (UTC+0)
Comment Actions
Environment
Event
Status
Beta-Cluster
web_ab_test_enrollment
Verified
test.wikipedia.org
mediawiki.web_ui_scroll
Verified
test.wikipedias.org
editattemptstep
Verified
Jdlrobson
subscribed.
May 16 2023, 10:55 PM
2023-05-16 22:55:16 (UTC+0)
Comment Actions
I checked the patches, and we're good to deploy as follows:
Wed:
Thur:
gerritbot
added a comment.
May 17 2023, 8:14 PM
2023-05-17 20:14:36 (UTC+0)
Comment Actions
Change 920386
merged
by jenkins-bot:
[operations/mediawiki-config@master] Enable zebra ab test in hewiki
Stashbot
mentioned this in
T317375: [Config change] Deploy New Topic Tool as opt-out preference at fi.wiki (desktop)
May 17 2023, 8:17 PM
2023-05-17 20:17:30 (UTC+0)
Stashbot
mentioned this in
T308134: Deploy "add a link" to 9th round of wikis
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2023-05-17T20:17:29Z]
T308134
)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (
T317375
)]], [[gerrit:920386|Enable zebra ab test in hewiki (
T335972
)]]
Stashbot
added a comment.
May 17 2023, 8:19 PM
2023-05-17 20:19:06 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2023-05-17T20:19:04Z]
T308134
)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (
T317375
)]], [[gerrit:920386|Enable zebra ab test in hewiki (
T335972
)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.
Stashbot
added a comment.
May 17 2023, 8:29 PM
2023-05-17 20:29:09 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2023-05-17T20:29:06Z]
T308134
)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (
T317375
)]], [[gerrit:920386|Enable zebra ab test in hewiki (
T335972
)]] (duration: 11m 36s)
gerritbot
added a comment.
May 17 2023, 9:36 PM
2023-05-17 21:36:38 (UTC+0)
Comment Actions
Change 920793 had a related patch set uploaded (by Jdrewniak; author: Jdrewniak):
[operations/mediawiki-config@master] Enable Veector AB test on beta spanish wiki
gerritbot
added a comment.
May 18 2023, 12:59 PM
2023-05-18 12:59:30 (UTC+0)
Comment Actions
Change 920793
merged
by jenkins-bot:
[operations/mediawiki-config@master] [Beta] Enable Vector AB test on beta spanish wiki
LGoto
edited projects, added
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 4)
; removed
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 3)
May 22 2023, 5:20 PM
2023-05-22 17:20:53 (UTC+0)
LGoto
moved this task from
Incoming
to
Blocked on Others
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 4)
board.
gerritbot
added a comment.
May 23 2023, 3:19 PM
2023-05-23 15:19:40 (UTC+0)
Comment Actions
Change 922564 had a related patch set uploaded (by Jdrewniak; author: Jdrewniak):
[operations/mediawiki-config@master] Enable Vector "Zebra" AB test on Hebrew wikipedia
gerritbot
added a comment.
May 24 2023, 10:27 PM
2023-05-24 22:27:55 (UTC+0)
Comment Actions
Change 922939 had a related patch set uploaded (by Jdrewniak; author: Jdrewniak):
[mediawiki/skins/Vector@master] Override mw.experiments.getBucket() via body classes
gerritbot
added a comment.
May 24 2023, 11:55 PM
2023-05-24 23:55:03 (UTC+0)
Comment Actions
Change 922939
merged
by jenkins-bot:
[mediawiki/skins/Vector@master] Override mw.experiments.getBucket() via body classes
ReleaseTaggerBot
added a project:
MW-1.41-notes (1.41.0-wmf.11; 2023-05-30)
May 25 2023, 1:00 AM
2023-05-25 01:00:41 (UTC+0)
gerritbot
added a comment.
May 25 2023, 6:39 PM
2023-05-25 18:39:00 (UTC+0)
Comment Actions
Change 923409 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):
[mediawiki/skins/Vector@master] Use document feature classes to extract A/B test state
gerritbot
added a comment.
May 25 2023, 7:18 PM
2023-05-25 19:18:20 (UTC+0)
Comment Actions
Change 923281 had a related patch set uploaded (by Jdrewniak; author: Jdlrobson):
[mediawiki/skins/Vector@wmf/1.41.0-wmf.10] Use document feature classes to extract A/B test state
gerritbot
added a comment.
May 25 2023, 7:23 PM
2023-05-25 19:23:23 (UTC+0)
Comment Actions
Change 923409
merged
by jenkins-bot:
[mediawiki/skins/Vector@master] Use document feature classes to extract A/B test state
Jdlrobson
moved this task from
Blocked on Others
to
Code Review
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 4)
board.
May 25 2023, 7:29 PM
2023-05-25 19:29:51 (UTC+0)
Comment Actions
@KSarabia-WMF
will backport
to 1.41.0-wmf.10 today (Thursday 25th May) and we will aim to roll this out again early next week.
gerritbot
added a comment.
May 25 2023, 8:20 PM
2023-05-25 20:20:27 (UTC+0)
Comment Actions
Change 923281
merged
by jenkins-bot:
[mediawiki/skins/Vector@wmf/1.41.0-wmf.10] Use document feature classes to extract A/B test state
Stashbot
added a comment.
May 25 2023, 8:21 PM
2023-05-25 20:21:07 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2023-05-25T20:21:06Z]
T335972
)]]
Stashbot
added a comment.
May 25 2023, 8:22 PM
2023-05-25 20:22:36 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2023-05-25T20:22:36Z]
T335972
)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet
Stashbot
added a comment.
May 25 2023, 8:32 PM
2023-05-25 20:32:05 (UTC+0)
Comment Actions
Mentioned in SAL (#wikimedia-operations)
[2023-05-25T20:32:04Z]
T335972
)]] (duration: 10m 58s)
ReleaseTaggerBot
edited projects, added
MW-1.41-notes (1.41.0-wmf.10; 2023-05-23)
; removed
MW-1.41-notes (1.41.0-wmf.11; 2023-05-30)
May 25 2023, 9:00 PM
2023-05-25 21:00:34 (UTC+0)
KSarabia-WMF
added subscribers:
NBaca-WMF
sgrabarczuk
May 25 2023, 11:10 PM
2023-05-25 23:10:08 (UTC+0)
Comment Actions
FYI - This is the revised Zebra A/B Test rollout timeline:
Enable a/b test for the following wikis on
Wed May 17th
Monday 5/29: hewiki
Enable a/b test for the following wikis on
Thur May 18th
Tuesday 5/30: enwiki, fawiki, frwiki, kowiki, ptwiki, srwiki, thwiki, trwiki, viwiki
cc:
@jwang
@sgrabarczuk
@NHillard-WMF
KSarabia-WMF
updated the task description.
(Show Details)
May 25 2023, 11:11 PM
2023-05-25 23:11:10 (UTC+0)
KSarabia-WMF
added a comment.
May 26 2023, 5:53 PM
2023-05-26 17:53:25 (UTC+0)
Comment Actions
Update: 5/29 is a US holiday and there are no backports that day!
The following is hopefully the
final
revised timeline.
Enable a/b test for the following wikis on Tuesday 5/30: hewiki
Enable a/b test for the following wikis on Wednesday 5/31: enwiki, fawiki, frwiki, kowiki, ptwiki, srwiki, thwiki, trwiki, viwiki
cc:
@jwang
@sgrabarczuk
@NHillard-WMF
jwang
mentioned this in
T333222: Create instrumentation spec for Zebra #9 prototype
May 30 2023, 9:54 PM
2023-05-30 21:54:03 (UTC+0)
Jdrewniak
mentioned this in
T327718: Increase space between groups in main menu and page tools menu
Jun 1 2023, 4:19 PM
2023-06-01 16:19:42 (UTC+0)
Jdrewniak
moved this task from
Code Review
to
Ready for Signoff
on the
Web-Team-Backlog-Archived (Web Team FY2022-23 Q4 Sprint 4)
board.
Jun 1 2023, 4:41 PM
2023-06-01 16:41:03 (UTC+0)
KSarabia-WMF
added a subtask:
T337956: Turn off Zebra A/B Test
Edited
Jun 1 2023, 4:44 PM
2023-06-01 16:44:59 (UTC+0)
Comment Actions
Turn off test
and put that ticket in
Sign Off
by 6/16
KSarabia-WMF
updated the task description.
(Show Details)
Jun 1 2023, 4:47 PM
2023-06-01 16:47:16 (UTC+0)
jwang
added a comment.
Jun 1 2023, 9:01 PM
2023-06-01 21:01:10 (UTC+0)
Comment Actions
@KSarabia-WMF
, Is it possible to prolong the AB test by an additional 3 or 4 days? This way, we would obtain a full two weeks of data once the edit button issue is resolved.
KSarabia-WMF
added a comment.
Jun 2 2023, 10:44 PM
2023-06-02 22:44:48 (UTC+0)
Comment Actions
In
T335972#8897405
@jwang
wrote:
@KSarabia-WMF
, Is it possible to prolong the AB test by an additional 3 or 4 days? This way, we would obtain a full two weeks of data once the edit button issue is resolved.
Yes, i'll move the deadline
KSarabia-WMF
updated the task description.
(Show Details)
Jun 2 2023, 10:50 PM
2023-06-02 22:50:19 (UTC+0)
KSarabia-WMF
updated the task description.
(Show Details)
Jun 2 2023, 10:55 PM
2023-06-02 22:55:41 (UTC+0)
Alpyne
awarded a token.
Jun 3 2023, 1:01 PM
2023-06-03 13:01:49 (UTC+0)
LGoto
reassigned this task from
KSarabia-WMF
to
jwang
Jun 5 2023, 5:25 PM
2023-06-05 17:25:17 (UTC+0)
LGoto
added a subscriber:
KSarabia-WMF
jwang
closed this task as
Resolved
Jun 5 2023, 7:33 PM
2023-06-05 19:33:44 (UTC+0)
Comment Actions
The ab test events are available in
mediawiki_web_ab_test_enrollment
with experiment name
skin-vector-zebra-experiment
now.
Jdlrobson
awarded a token.
Jun 8 2023, 2:54 PM
2023-06-08 14:54:20 (UTC+0)
Comment Actions
Congrats on the milestone here!
bwang
closed subtask
T336969: [Zebra AB test] Fix the mixing of global and user IDs for AB Test Enrollment Bucketing
as
Resolved
Jun 8 2023, 5:35 PM
2023-06-08 17:35:46 (UTC+0)
Jdlrobson
closed subtask
T337956: Turn off Zebra A/B Test
as
Resolved
Jun 29 2023, 6:01 PM
2023-06-29 18:01:11 (UTC+0)
gerritbot
added a comment.
Aug 15 2023, 2:34 PM
2023-08-15 14:34:52 (UTC+0)
Comment Actions
Change 922564
abandoned
by Jdrewniak:
[operations/mediawiki-config@master] Enable Vector "Zebra" AB test on Hebrew wikipedia
Reason:
gerritbot
added a comment.
Aug 24 2023, 2:47 PM
2023-08-24 14:47:29 (UTC+0)
Comment Actions
Change 918568
abandoned
by Jdrewniak:
[operations/mediawiki-config@master] Launch content separation Zebra AB Test
Reason:
ovasileva
closed subtask
T336299: QA A/B test schema for Zebra #9 A/B test
as
Resolved
Oct 17 2023, 5:58 PM
2023-10-17 17:58:42 (UTC+0)
gerritbot
added a comment.
May 20 2024, 6:25 PM
2024-05-20 18:25:06 (UTC+0)
Comment Actions
Change #918613
abandoned
by Jdlrobson:
[mediawiki/skins/Vector@master] Make ABRequirement aware of Require
Reason:
Abandoning since associated phabricator ticket is resolved.
Maintenance_bot
removed a project:
Patch-For-Review
May 20 2024, 6:31 PM
2024-05-20 18:31:58 (UTC+0)
Log In to Comment
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct.
Wikimedia Foundation
Code of Conduct
Disclaimer
CC-BY-SA
GPL
Credits
US