diff --git a/thesis/2-Background.tex b/thesis/2-Background.tex index 560bb498fed8cebd05f22de386f32a8087c27844..6a571ba2afec5c612632233fe4b4678c0a097b27 100644 --- a/thesis/2-Background.tex +++ b/thesis/2-Background.tex @@ -15,12 +15,12 @@ They enabled efficient patrolling of articles by users with little to no knowled Thanks to Wikipedia's idiosyncratic software architecture, this is possible even in the most ``manual'' quality control work (i.e. using watchlists to patrol articles): Representing information changes via diffs allows editors to quickly spot content that deviates from its immediate context~\cite{GeiRib2010}. -Others were worried it was getting increasingly intransparent how the encyclopedia functions and not only ``[k]eeping traces obscure help[ed] the powerful to remain in power''~\cite{ForGei2012}, +Others were worried it was getting increasingly untransparent how the encyclopedia functions and not only ``[k]eeping traces obscure help[ed] the powerful to remain in power''~\cite{ForGei2012}, but entry barriers for new users were gradually set higher~\cite{HalGeiMorRied2013}: They had to learn to interact with a myriad of technical tools, learn wiki syntax, but also navigate their ground in a complex system with a decentralised socio-technical mode of governance~\cite{Geiger2017}. Ford and Geiger even cite a case in which an editor was not sure whether a person deleted their articles or a bot~\cite{ForGei2012}. -Quality control mechanisms on Wikipedia can be categorised into following three groups according to their level of automatisation: fully automated, semi-automated, and manual. +Quality control mechanisms on Wikipedia can be categorised into following three groups according to their level of automation: fully automated, semi-automated, and manual. Fully automated tools include bots, the edit filters, which are the focus of the present thesis, and other MediaWiki's features such as the mechanism for page protection~\cite{Mediawiki:PageProtection} (which allows restricting editing of a particular page to certain usergroups), and title~\cite{Mediawiki:TitleBlacklist} and spam~\cite{Mediawiki:SpamBlacklist} blacklists which operate on regular expression basis to disallow specific titles or publication of some links perceived as spam. There is also the automatically functioning Wikipedian machine learning framework ORES~\cite{ORES} which computes quality scores per article or revision. @@ -52,7 +52,7 @@ However, the ones the present work focuses on—the rapid response anti-vandalis Different aspects of bots and their involvement in quality control have been investigated: In the paper referenced above, the researchers employ their method of trace ethnography (more on it in chapter~\ref{chap:methods}) to follow a disrupting editor around Wikipedia and comprehend the measures taken in collaboration by bots (ClueBot~\cite{Wikipedia:ClueBot} and HBC AIV helperbot7~\cite{Wikipedia:HBCAIVHelperbot}) as well as humans using semi-automated tools (Huggle~\cite{Wikipedia:Huggle} and Twinkle~\cite{Wikipedia:Twinkle}) up until they achieved that the malicious editor in question was banned~\cite{GeiRib2010}. Halfaker and Riedl offer a historical review of bots and semi-automated tools and their involvement in vandal fighting~\cite{HalRied2012}, assembling a comprehensive list of tools and touching on their working principle (rule vs. machine learning based). -They also develop a bot taxonomy classifying bots in one of the following three groups according to their task area: content injection, montioring or curating; augmenting MediaWiki functionality; and protection from malicious activity. +They also develop a bot taxonomy classifying bots in one of the following three groups according to their task area: content injection, monitoring or curating; augmenting MediaWiki functionality; and protection from malicious activity. In~\cite{GeiHal2013}, Geiger and Halfaker conduct an in-depth analysis of ClueBot NG, ClueBot's machine learning based successor, and its place within Wikipedia's vandal fighting infrastructure concluding that quality control on Wikipedia is a robust process and most malicious edits eventually get reverted even with some of the actors (temporaly) inactive, although at a different speed. They discuss the mean times to revert of different mechanisms, their observations coinciding with figure~\ref{fig:funnel-no-filters}, and also comment on the (un)realiability of external infrastructure bots rely upon (run on private computers, which causes downtimes). @@ -68,7 +68,7 @@ Very crucial for the current analysis will also be Livingstone's observation in If these things are not in the software, an external bot could do them. [...] The main difference is where it runs and who runs it''~\cite{Livingstone2016}. This thought is also scrutinised by Geiger~\cite{Geiger2014} who examines in detail what the difference and repercussions are of code that is part of the core software and code that runs alongside it (such as bots) which he calls ``bespoke code''. -Geiger pictures Wikipedia as a big socio-technical assemblage of software pieces and social processes, often completely intransparent for an outside observer who is not able to identify the single components of this system and how they interact with one another to provide the end result to the public. +Geiger pictures Wikipedia as a big socio-technical assemblage of software pieces and social processes, often completely untransparent for an outside observer who is not able to identify the single components of this system and how they interact with one another to provide the end result to the public. He underlines that components which are not strictly part of the server-side codebase but run by various volunteers (which is well true for the most parts of Wikipedia, it is a community project) on their private infrastructure constitute the major part of Wikipedia and also that they can experience a downtime at any moment. %TODO this may have been largely true in 2014, but there is a trend towards centralisation (bots run on the toolserver, etc). The vital tasks they perform, such as vandalism fighting, are often taken for granted, much to their developers' aggravation. @@ -129,7 +129,7 @@ Geiger and Ribes~\cite{GeiRib2010} touch on Lupin's Anti-vandal tool~\cite{Wikip Halfaker and Riedl talk about VandalProof~\cite{HalRied2012}. Some of these tools are more automated than others: Huggle and STiki for instance are able to revert an edit, issue a warning to the offending editor, and post a report on the AIV dashboard (if the user has already exhausted the warning limit) upon a single click. -The javascript based browser extension Twinkle on the other hand adds contextual links to other parts of Wikipedia which facilitates fulfilment of particular tasks such as rollback multiple edits, report problematic users to AIV, or nominate an article for deletion~\cite{GeiRib2010}. +The javascript based browser extension Twinkle on the other hand adds contextual links to other parts of Wikipedia which facilitates fulfillment of particular tasks such as rollback multiple edits, report problematic users to AIV, or nominate an article for deletion~\cite{GeiRib2010}. The main feature of Huggle and STiki is that they both compile a central queue of potentially harmful edits for all their users to check. The difference between both programs are the heuristics they use for their queues: By default, Huggle sends edits by users with warnings on their user talk page to the top of the queue, places edits by IP editors higher and ignores edits made by bots and other Huggle users altogether\cite{GeiRib2010}. @@ -141,11 +141,11 @@ Some critique that has been voiced regarding semi-automated anti-vandalism tools The concern is that some of the users of said tools see themselves as vandal fighters on a mission to slay the greatest number of monsters (vandals) possible and by doing so to excell in the ranks \footnote{STiki actually has a leader board: \url{https://en.wikipedia.org/w/index.php?title=Wikipedia:STiki/leaderboard&oldid=905145147}}. This is for one a harmful way to view the project, neglecting the ``assume good faith'' guideline~\cite{Wikipedia:GoodFaith} -and also leads to such users seeking out easy to judge instancies from the queues in order to move onto the next entry more swiftly and gather more points +and also leads to such users seeking out easy to judge instances from the queues in order to move onto the next entry more swiftly and gather more points leaving more subtle cases which really require human judgement to others. Transparency wise, one can criticise that the heuristics they use to compile the queues of potential malicious edits in need of attention are oftentimes obfuscated by the user interface and so the editors using them are not aware why exactly these and not other edits are displayed to them. -The heurisics to use are configurable to an extent, however, one needs to be aware of this option~\cite{GeiRib2010}. %TODO maybe move to pitfalls/concerns discussion +The heuristics to use are configurable to an extent, however, one needs to be aware of this option~\cite{GeiRib2010}. %TODO maybe move to pitfalls/concerns discussion %TODO review this concern as well! diff --git a/thesis/3-Methods.tex b/thesis/3-Methods.tex index 3cc1ae00faf5ed6a133b113a765a6916ee705c6a..ca22bd5b4d64b0ddeb0031e087e6a004b22d12f1 100644 --- a/thesis/3-Methods.tex +++ b/thesis/3-Methods.tex @@ -10,9 +10,9 @@ The emergent coding introduced in section~\ref{sec:gt}, combined with trace ethn %\section{Open Science} The whole work tries to adhere to the principles of open science and reproducible research. -According to the definition of Bartling and Friesike provided in their book \textit{Opening Science}~\cite{BarFri2014}, open science is primarily characterised, unsurpisingly, by its openness. +According to the definition of Bartling and Friesike provided in their book \textit{Opening Science}~\cite{BarFri2014}, open science is primarily characterised, unsurprisingly, by its openness. There is an open communication of the methods and results in every stage of the research project, allowing, importantly, for an easier disclosure of negative results. -The code for all data processing and computational anlyses I have done, as well as other artefacts I have used or compiled have been openly accessible in the project's repository since the beginning of the present research~\cite{gitlab} +The code for all data processing and computational analyses I have done, as well as other artefacts I have used or compiled have been openly accessible in the project's repository since the beginning of the present research~\cite{gitlab} and can be re-used under a free license. %TODO (which one?) Anyone interested can follow the process and/or use the data or scripts to verify my computations or run their own and thus continue this research along one of the directions suggested in section~\ref{sec:further-studies} or in a completely new one. diff --git a/thesis/4-Edit-Filters.tex b/thesis/4-Edit-Filters.tex index 6adb8ad32c6d8af5e2d84707664c86939fd2db69..97c9ca6e7b8525ad7a79cab2cb59bf84cbe41363 100644 --- a/thesis/4-Edit-Filters.tex +++ b/thesis/4-Edit-Filters.tex @@ -74,7 +74,7 @@ And the currently configured filter actions are: ``disallow''. So, if a user whose status is not confirmed \footnote{A confirmed user can do following five things a non-confirmed user cannot: create pages; move pages; edit semi-protected pages; upload files; vote in certain elections (a different minimum edit count can be required for certain elections). -An account can be explicitely confirmed, most accounts are autoconfirmed though. +An account can be explicitly confirmed, most accounts are autoconfirmed though. Generally, accounts are autoconfirmed when they have been registered for at least four days and have made a minimum of ten edits~\cite{Wikipedia:About}. The requirements are adjustable.} yet tries to edit a page in the article namespace which contains ``Featured'' or ``Good article'' and they either insert a redirect, delete 3/4 of the content or add 3/4 on top, the edit is automatically disallowed. @@ -103,10 +103,10 @@ and actions to take when the filter's pattern matches. %************************************************************************ -\section[The AbuseFilter MediaWiki Extension]{The AbuseFilter\footnote{Note that the user facing elements of this extention were renamed to ``edit filter'', however the extension itself, as well as its corresponding permissions, database tables etc. still reflect the original name.} Mediawiki Extension} +\section[The AbuseFilter MediaWiki Extension]{The AbuseFilter\footnote{Note that the user facing elements of this extension were renamed to ``edit filter'', however the extension itself, as well as its corresponding permissions, database tables etc. still reflect the original name.} Mediawiki Extension} \label{sec:mediawiki-ext} -At the end, from a technical perspective, Wikipedia's edit filters are a MediaWiki plugin that allows every edit (and some other editor's actions) to be checked against a speficied pattern before it is published. +At the end, from a technical perspective, Wikipedia's edit filters are a MediaWiki plugin that allows every edit (and some other editor's actions) to be checked against a specified pattern before it is published. The extension introduces following database tables where all data generated by it is stored: \emph{abuse\_filter}, \emph{abuse\_filter\_log}, \emph{abuse\_filter\_action} and \emph{abuse\_filter\_history} \cite{gerrit-abusefilter-tables}. \emph{abuse\_filter} contains detailed information about every filter. @@ -127,7 +127,7 @@ The plugin defines following possible filter actions: The documentation of the AbuseFilter extension provides us comprehensive definitions for these~\cite{Mediawiki:AbuseFilterActions}: \begin{itemize} \item \emph{tag}: The contribution is tagged with a specific tag (which can be defined and styled by the edit filter manager) which then appears on Recent Changes, contributions, logs, history pages, etc. and allows aggregations of lists for dashboards and similar. - \item \emph{throttle}: The filter is activated upon the tripping of a rate limit. Configurable parameters are the allowed number of actions, the period of time in which these actions must occur, and how those actions are grouped. Actions can be grouped by user, IP address, /16 IP range, creation date of the user account, page, site, the edit count of the user or a combination thereof. (A simple example for trottling is something like ``do this if page X is edited more than Y times in Z seconds''.) + \item \emph{throttle}: The filter is activated upon the tripping of a rate limit. Configurable parameters are the allowed number of actions, the period of time in which these actions must occur, and how those actions are grouped. Actions can be grouped by user, IP address, /16 IP range, creation date of the user account, page, site, the edit count of the user or a combination thereof. (A simple example for throttling is something like ``do this if page X is edited more than Y times in Z seconds''.) \item \emph{warn}: A warning is displayed that the edit may not be appreciated. (The warning message is configurable by the edit filter manager.) The editor who tripped the filter is provided with the opportunity to revise their edit and re-submit it. A link to the false positives page~\cite{Wikipedia:EditFilterFalsePositives} is also provided. \item \emph{blockautopromote}: The user whose action matched the filter's pattern is banned from receiving extra groups from \emph{\$wgAutopromote} for a random period of 3 to 7 days. %TODO what is wgAutopromote? \item \emph{block}: The user who triggered the filter is blocked indefinitely. An error message is displayed to inform the user of this action. @@ -184,13 +184,13 @@ The discussion reflects a mix of ideological and practical concerns. The biggest controversies lay along the lines of filters being public vs. private (hidden from public view) \footnote{The terms ``private'' and ``hidden'' are used interchangeably for such filters throughout the thesis.} and the actions the filters were to invoke upon a match. -An automated rights revokation or a block of the offending editor with no manual confirmation by a real person were of particular concern to a lot of editors (they were worried that the filters would not be able to understand context thus resulting in too many false positives and blocking many legitimate edits and editors). +An automated rights revocation or a block of the offending editor with no manual confirmation by a real person were of particular concern to a lot of editors (they were worried that the filters would not be able to understand context thus resulting in too many false positives and blocking many legitimate edits and editors). As far as I understood, these features were technically implemented but never really used on English Wikipiedia. As to the public vs. private debate, the initial plan was that all filters are hidden from public view and only editors with special permissions (the edit filter managers) were supposed to be able to view and modify the patterns and consult the logs. The core developer of the extension was reasoning that its primary purpose was to fend off really persistent vandals with reasonable technical understanding who were ready to invest time and effort to circumvent anti-vandal measures -and that it was therefore unwise to make circumvention easier to them by allowing them to view the pattern according to which their edits were supressed. -This was however met with serious resistence by the community who felt that such secret extension was contradicting Wikipedia's values of openness and transparency. +and that it was therefore unwise to make circumvention easier to them by allowing them to view the pattern according to which their edits were suppressed. +This was however met with serious resistance by the community who felt that such secret extension was contradicting Wikipedia's values of openness and transparency. Besides, opponents of the filters being completely private were concerned that the tool was quite powerful and hiding everything will prevent the community from monitoring for errors and abuse. Related to the above discussion, there was also disagreement regarding who was to have access to the newly developed tool. @@ -211,7 +211,7 @@ Here, User:Werdna was reasoning that the Edit Filters allow for fine-tuning of s Although there were some diverging opinions on what the extension was supposed to target, in a nutshell, the motivation for its introduction seems to have been as follows: Bots weren't reverting some kinds of vandalism fast enough, or, respectively, these vandalism edits required a human intervention and took more than a single click to get reverted. These were mostly obvious but pervasive cases of vandalism (e.g. moving a lot of pages to some nonsensical name), possibly introduced in a (semi-)automated fashion, that took some time and effort to clean up. -The motivation of the extention's developers was that if a filter just disallows such vandalism, vandal fighters could use their time more productively and check less obvious cases for which more background knowledge/context is needed in order to decide whether an edit is vandalism or not. +The motivation of the extension's developers was that if a filter just disallows such vandalism, vandal fighters could use their time more productively and check less obvious cases for which more background knowledge/context is needed in order to decide whether an edit is vandalism or not. According to the discussion archives, following types of edits were supposed to be targeted by the extension:\\ \url{https://en.wikipedia.org/wiki/Special:Contributions/Omm_nom_nom_nom}\\ \url{https://en.wikipedia.org/wiki/Special:Contributions/AV-THE-3RD}\\ @@ -234,7 +234,7 @@ The Edit Filter Requested page asks users to go through the following checklist \item problems with a single page are not suitable for an edit filter, since filters are applied to all edits; \item filters, after adding up, make editing slower, so the usefulness of every single filter and condition has to be carefully considered; \item in depth checks should be done by a separate software that users run on their own machines; - \item no trivial errors should be catched by filters (e.g. concerning style guidelines); + \item no trivial errors should be caught by filters (e.g. concerning style guidelines); \item there are the Titles Blacklist~\cite{Mediawiki:TitleBlacklist} and the Link/Spam Blacklist~\cite{Mediawiki:SpamBlacklist} which should be used if the issue at hand has to do with a problematic title or link. \end{itemize} @@ -368,12 +368,12 @@ As shown elsewhere~\cite{HalGeiMorRied2013}, this shift had a lot of repercussio 2006 & BAG was first formed \\ 13 Mar 2006 & 1st version of Bots/Requests for approval is published: some basic requirements (also valid today) are recorded \\ 28 Jul 2006 & VoABot II ("In the case were banned users continue to use sockpuppet accounts/IPs to add edits clearly rejected by consensus to the point were long term protection is required, VoABot may be programmed to watch those pages and revert those edits instead. Such edits are considered blacklisted. IP ranges can also be blacklisted. This is reserved only for special cases.") \\ -21 Jan 2007 & Twinkle Page is first published (empty), filled with a basic description by beginings of Feb 2007 \\ +21 Jan 2007 & Twinkle Page is first published (empty), filled with a basic description by beginnings of Feb 2007 \\ 24 Jul 2007 & Request for Approval of original ClueBot \\ 16 Jan 2008 & Huggle Page is first published (empty) \\ 18 Jan 2008 & Huggle Page is first filled with content \\ -23 Jun 2008 & 1st version of Edit Filter page is published: User:Werdna announces they're currently developing the extention \\ - 2 Oct 2008 & \url{https://en.wikipedia.org/wiki/Wikipedia_talk:Edit_filter} was first archived; its last topic was the voting for/against the extention which seemed to have ended end of Sep 2008 \\ +23 Jun 2008 & 1st version of Edit Filter page is published: User:Werdna announces they're currently developing the extension \\ + 2 Oct 2008 & \url{https://en.wikipedia.org/wiki/Wikipedia_talk:Edit_filter} was first archived; its last topic was the voting for/against the extension which seemed to have ended end of Sep 2008 \\ Jun 2010 & STiki initial release \\ 20 Oct 2010 & ClueBot NG page is created \\ 11 Jan 2015 & 1st commit to github ORES repository \\ @@ -431,13 +431,13 @@ The numbers also demonstrate that this is the most exclusive group: as mentioned in section~\ref{section:who-can-edit}, there are currently 154 edit filter managers on EN Wikipedia, compared to at least 232 bot operators~\cite{Wikipedia:BotOperators} (most likely not all bot operators are listed in the category~\cite{Wikipedia:FAQCategory}) and 6130 users who have the \emph{rollback} permission~\cite{Wikipedia:Rollback}. -As to the difficulty/compteneces needed, it is probably easiest to learn to use semi-automated tools where one ``only'' has to master the user interface of the software. -Bots require presumably most background knowledge since one has to not only be familiar with a programming langauage but also learn to interact with Wikipedia's API, etc. +As to the difficulty/competences needed, it is probably easiest to learn to use semi-automated tools where one ``only'' has to master the user interface of the software. +Bots require presumably most background knowledge since one has to not only be familiar with a programming language but also learn to interact with Wikipedia's API, etc. Filters on the other hand, are arguably easier to use: here, ``only'' understanding of regular expressions is required. %Different pitfalls and concerns are express As already summarised in chapter~\ref{chap:background}, critical voices worry about various aspects of the individual quality control mechanisms (see also table~\ref{table:mechanisms-comparison}). -Concerns with filters resemble somewhat the concerns expressed about bots: namely, the apprehension of a fully-automated mechanism taking (potentially erroneuos) decisions about excluding editors from participation. +Concerns with filters resemble somewhat the concerns expressed about bots: namely, the apprehension of a fully-automated mechanism taking (potentially erroneous) decisions about excluding editors from participation. In consequence, community consensus on using filter actions such as \emph{rangeblock}, \emph{block}, and \emph{degroup} never happened. According to the discussion archives~\cite{Wikipedia:EditFilterTalkArchive1}, others feared that edit filters were placing a lot of power in the hands of very few people. diff --git a/thesis/5-Overview-EN-Wiki.tex b/thesis/5-Overview-EN-Wiki.tex index d710e62957c3d9f99b3ad84f73cdbc6a3ac9f7d8..d8b52807c78610800ee5d6a671058a6a29df1164 100644 --- a/thesis/5-Overview-EN-Wiki.tex +++ b/thesis/5-Overview-EN-Wiki.tex @@ -96,8 +96,8 @@ One feature which guided me here was the filter action which represents the judg Since communication is crucial when assuming good faith, all ambiguous cases which have a less ``grave'' filter action such as ``tag'' or ``warn'' (which seeks to give feedback and thereby effect/influence a constructive contribution) have received a ``good\_faith'' label. On the other hand, filters set to ``disallow'' were tagged as ``vandalism'' or a particular type thereof, since the filter action is a clear sign that at least the edit filter managers have decided that seeking a dialog with the offending editor is no longer an option. -For the second round of labeling, I tagged the whole dataset again using the compiled code book (see \ref{app:code_book}) and assigned to every filter exactly one label—the one deemed most appropriate (although oftentimes alternative possibilites were listed as notes), without looking at the labels I assigned the first time around. -I intended to compare the labels from both coding sessions and focus on more ambiguous cases, re-evaluting them using all available information (patterns, public comments, labels from both sessions, as well as any notes I made along the line). +For the second round of labeling, I tagged the whole dataset again using the compiled code book (see \ref{app:code_book}) and assigned to every filter exactly one label—the one deemed most appropriate (although oftentimes alternative possibilities were listed as notes), without looking at the labels I assigned the first time around. +I intended to compare the labels from both coding sessions and focus on more ambiguous cases, re-evaluating them using all available information (patterns, public comments, labels from both sessions, as well as any notes I made along the line). Unfortunately, time was scarce, so the analysis of the present section is based upon the second round of labeling. Comparing codes from both labeling sessions and refining the coding, or respectively have another person label the data should be done in the future. @@ -119,14 +119,14 @@ For these, from the edit alone there is no way of knowing whether the deletion w \subsection{Vandalism} The vast majority of edit filters on EN Wikipedia could be said to target (different forms of) vandalism, i.e. maliciously intended disruptive editing (or other activity)~\cite{Wikipedia:Vandalism}. -Some examples thereof are filters for juvenile types of vandalism (inserting swear or obscene words or nonsence sequences of characters into articles), for hoaxing (inserting obvious or less obvious false information in articles), for template vandalism (modifying a template in a disruptive way which is quite severe, since templates are displayed on various pages), or for spam (inserting links to promotional content, often not related to the content being edited). %TODO stick to one terminology; no "juvenile" vandalism otherwise +Some examples thereof are filters for juvenile types of vandalism (inserting swear or obscene words or nonsense sequences of characters into articles), for hoaxing (inserting obvious or less obvious false information in articles), for template vandalism (modifying a template in a disruptive way which is quite severe, since templates are displayed on various pages), or for spam (inserting links to promotional content, often not related to the content being edited). %TODO stick to one terminology; no "juvenile" vandalism otherwise All codes belonging to the vandalism category together with a definition and examples can be consulted in the code book attached in the appendix~\ref{app:code_book}. Some vandalism types seem to be severer than others (e.g. sock puppetry \footnote{Sock puppetry denotes the creation and employment of several accounts for various purposes such as pushing a point of view, or circumventing bans. For more information, see the code book in the appendix~\ref{app:code_book}} or persistent long term vandals). It is mostly in these cases that the implemented filters are hidden. -Labels refering to such types of vandalism form their own subcategory: ``hardcore vandalism''. %TODO think about naming +Labels referring to such types of vandalism form their own subcategory: ``hardcore vandalism''. %TODO think about naming It should be mentioned at this point that I also classified ``harassment'' and ``personal attacks'' as ``hardcore vandalism'', since these types of edits are highly harmful and often dealt with by hidden filters, although according to~\cite{Wikipedia:Vandalism} both behaviours are disruptive editing rather than vandalism and should generally be handled differently. \subsection{Good Faith} @@ -148,7 +148,7 @@ These filters form the ``maintenance'' category. Some of them target for instance bugs like broken syntax caused by a faulty browser extension. Or there are such which simply track particular behaviours (such as mobile edits or edits made by unflagged bots) for various purposes. -The ``maintenance'' category differs conceptually from the ``vandalism'' and ``good faith'' ones in so far that the logic behind it isn't editors' intention, but rather ``side''-occurances that mostly went wrong. +The ``maintenance'' category differs conceptually from the ``vandalism'' and ``good faith'' ones in so far that the logic behind it isn't editors' intention, but rather ``side''-occurrences that mostly went wrong. I've also grouped here various test filters (used by individual editors or jointly used by all editors). @@ -163,7 +163,7 @@ used to code all filters where the functionality stayed completely opaque for th \section{Filter Characteristics} \label{sec:patterns} -This section explores some general features of the edit filters on Engish Wikipedia based on the data from the \emph{abuse\_filter} table. +This section explores some general features of the edit filters on English Wikipedia based on the data from the \emph{abuse\_filter} table. The scripts that generate the statistics discussed here, can be found in the jupyter notebook in the project's repository~\cite{gitlab}. @@ -173,7 +173,7 @@ As of 6 January 2019 there are $954$ filters in the \emph{abuse\_filter} table. It should be noted, that if a filter gets deleted, merely a flag is set to indicate so, but no entries are removed from the database. So, the above mentioned $954$ filters are all filters ever made up to this date. This doesn't mean that it never changed what the single filters are doing, since edit filter managers can freely modify filter patterns, so at some point a filter could be doing one thing and in the next moment it can be filtering a completely different phenomenon. -There are cases of filters being ``repurposed'' or modified to filter for example a more general occurance. +There are cases of filters being ``re-purposed'' or modified to filter for example a more general occurrence. This doesn't happen very often though. Mostly, if a filter is not useful anymore, it is just disabled and eventually deleted and new filters are implemented for current problems. @@ -217,8 +217,8 @@ Another interesting parameter observed here are the currently configured filter Figure~\ref{fig:all-active-filters-actions} depicts the actions set up for all enabled filters. And figures~\ref{fig:active-public-actions} and~\ref{fig:active-hidden-actions} show the actions of all enabled public and hidden filters respectively. It is noticeable that the most common action for the enabled hidden filters is ``disallow'' whereas most enabled public filters are set to ``tag'' or ``tag,warn''. -This is congruent with the community's claim that hidden filters target particularly perstistent vandalism, which is best outright disallowed. -A lot of public filters on the other hand still assume good faith from the editors and try to dissuade them from engaging in disruptive behaviour by using warnings or just tag conspicious behaviour for further investigation. +This is congruent with the community's claim that hidden filters target particularly persistent vandalism, which is best outright disallowed. +A lot of public filters on the other hand still assume good faith from the editors and try to dissuade them from engaging in disruptive behaviour by using warnings or just tag conspicuous behaviour for further investigation. \begin{figure} \centering @@ -375,7 +375,7 @@ I've undertaken following steps in an attempt to verify or refute each of these \\ \textbf{The filter hits mirror the overall edits pattern from this time} \\ I've compared the filter hits pattern with the overall number of edits of the time (May 2015–May 2016). -No correspondance could be determined (see figure~\ref{fig:edits-development}). +No correspondence could be determined (see figure~\ref{fig:edits-development}). \\ \\ \textbf{There was a general rise in vandalism in this period}\\ @@ -409,8 +409,8 @@ Since so far neither of the other hypothesis could be verified, this explanation Another piece of data that seems to support it is the breakdown of the filter hits according to triggered filter action. As demonstrated on figure~\ref{fig:filter-hits-actions}, there was above all a significant hits peak caused by ``log only'' filters. As discussed in section~\ref{sec:introduce-a-filter}, it is an established praxis to introduce new filters in ``log only'' mode and only switch on additional filter actions after a monitoring period showed that the filters function as intended. -Hence, it is plausible that new filters in logging mode were introduced, which were then switched off after a significant number of false positives occured. -However, upon closer scritiny, this could not be confirmed. +Hence, it is plausible that new filters in logging mode were introduced, which were then switched off after a significant number of false positives occurred. +However, upon closer scrutiny, this could not be confirmed. The filters with greatest number of hits in the period January–March 2016 are mainly the most triggered filters of all times and nearly all of them have been around for a while in 2016. Also, no bug or a comparable incident with the software was found upon an inspection of the extension's issue tracker~\cite{phab-abusefilter-2015}, or commit messages of the commits to the software done during May 2015–May 2016~\cite{gerrit-abusefilter-source}. Moreover, no mention of the hits surge was found in the noticeboard~\cite{Wikipedia:EditFilterNoticeboard} and edit filter talk page archives~\cite{Wikipedia:EditFilterTalkArchive2016}. @@ -419,18 +419,18 @@ The in section~\ref{sec:filter-activity} mentioned condition limit has not chang The only somewhat interesting pattern that seems to shed some light on the matter is the breakdown of hits according to the editor's action which triggered them: There is an obvious surge in the attempted account creations in the period November 2015–May 2016 (see figure~\ref{fig:filter-hits-editors-actions}). As a matter of fact, this could also be the explanation for the peak of log only hits—the most frequently tripped filter for the period January–March 2016 is filter 527 ``T34234: log/throttle possible sleeper account creations''. -It is a throttle filter, with no further actions enabled, so everytime an edit matches its pattern, a ``log only'' entry is created in the abuse log. +It is a throttle filter, with no further actions enabled, so every time an edit matches its pattern, a ``log only'' entry is created in the abuse log. %it disallows every X attempt, only logging the rest of the account creations. %I think in its current form, it does not actually disallow anything, a ``disallow'' action should be enabled for this and the filter action is only 'throttle'; so in this form, it seems to simply log account creations And the 3rd most active filter is a ``log only'' filter as well: 650 ``Creation of a new article without any categories''. (It was neither introduced at the time, nor was there any major change in the filter pattern.) Together, filters 527 and 650 are responsible for over 60\% of the ``log only'' hits in every of the months January, February and March 2016. -Another idea that seemed worth persuing was to look into the editors who tripped filters and their corresponding edits. +Another idea that seemed worth pursuing was to look into the editors who tripped filters and their corresponding edits. For the period January–March 2016 there are some very active IP editors, the top of whom (with over $1.000$ hits) seemed to be engaging exclusively in the (probably automated) posting of spam links. Their edits however constitute some 1-3\% of all hits from the period which is insufficient to explain the peak \footnote{Upon closer examination, these edits all seemed to contain spam links about erectile dysfunction medication and their IP records pertained to a Russian registry. It is however possible that the offending editors were using a VPN or another proxy technology. -The speculations abouth the intent of the edits remain out of the scope of the present work.}. +The speculations about the intent of the edits remain out of the scope of the present work.}. \begin{comment} so the explanation ``it was viagra spam coming from Russian IPs'' is somewhat unsatisfactory. (Yes, it was viagra spam, and yes, a ``whois'' lookup proved them to really be Russian IPs. @@ -733,10 +733,10 @@ A further assumption that didn't carry into effect was that ``filters in this ex \footnote{Here, by ``trigger'' is meant that an editor's action will match a filter's pattern and set off the configured filter's action(s).} fewer times than once every few hours''~\cite{Wikipedia:EditFilterTalkArchive1}. As a matter of fact, a quick glance at the AbuseLog~\cite{Wikipedia:AbuseLog} confirms that there are often multiple filter hits per minute, -so the mechanism is used fairly actively, despite that its areas of application partially diverge from the ones intially conceived. +so the mechanism is used fairly actively, despite that its areas of application partially diverge from the ones initially conceived. In fact, the numbers of filter hits on EN Wikipedia are in the same order of magnitude as the revert numbers (compare figures~\ref{fig:filter-hits} and \ref{fig:reverts}). -Regarding the temporal filter activity trends, it was ascertained that a sudden peak took place in the end of 2015–beginnings of 2016, after which the overall filter hit numbers stayed higher than they used to be before this occurence. +Regarding the temporal filter activity trends, it was ascertained that a sudden peak took place in the end of 2015–beginnings of 2016, after which the overall filter hit numbers stayed higher than they used to be before this occurrence. Although there were some pointers towards what happened there: a surge in account creation attempts and possibly a big spam wave (the latter has to be verified in a systematic fashion), no really satisfying explanation of the phenomenon could be established. diff --git a/thesis/6-Discussion.tex b/thesis/6-Discussion.tex index 6748013b820f9b0a75f1a1e0a0f6239a9bdda1f8..c0fa67d740ca6842818463f090a5c3306036c5cf 100644 --- a/thesis/6-Discussion.tex +++ b/thesis/6-Discussion.tex @@ -72,22 +72,22 @@ With this premise in mind, one has to ask: Why are rule based mechanisms such as the edit filters still widely in use? Several explanations of this phenomenon sound plausible. For one, Wikipedia's edit filters are an established system which works and does its work reasonably well, so there is no need to change it (``never touch a running system''). -Secondly, it has been organically weaven in Wikipedia's quality control ecosystem. +Secondly, it has been organically woven in Wikipedia's quality control ecosystem. There were historical necessities to which it responded and people at the time believed the mechanism to be the right solution to the problem they had. We could ask why was it introduced in the first place when there were already other mechanisms. -Beside the specific instancies of disruptive behaviour stated by the community as motivation to implement the extension, +Beside the specific instances of disruptive behaviour stated by the community as motivation to implement the extension, a very plausible explanation here is that since Wikipedia is a volunteer project a lot of stuff probably happens because at some precise moment there are particular people who are familiar with some concrete technologies so they construct a solution using the technologies they are good at using (or want to use). Another interesting reflection is that rule based systems are arguably easier to implement and above all to understand by humans which is why they still enjoy popularity today. On the one hand, overall less technical knowledge is required in order to implement a single filter: An edit filter manager has to ``merely'' understand regular expressions. Bot development by contrast is a little more challenging: -A developer needs resonable knowledge of at least one programming language and on top of that has to make themself familiar with artefacts like the Wikimedia API. -Moreover, since regular expressions are still somewhat human readable and apprehensible unlike a lot of popular machine learning algorithms, it is easier to hold rule based systems and their developers accountable. +A developer needs reasonable knowledge of at least one programming language and on top of that has to make themself familiar with artefacts like the Wikimedia API. +Moreover, since regular expressions are still somewhat human readable and comprehensible unlike a lot of popular machine learning algorithms, it is easier to hold rule based systems and their developers accountable. Filters are a simple mechanism (simple to implement) that swiftly takes care of cases that are easily recognisable as undesirable. ML needs training data (which expensive), and it is not simple to implement. What is more, rule based mechanisms allow for a finer granularity of control: -An edit filter can define a rule to explicitely exclude particular malicious users from publishing, which cannot be straightforwardly implemented in a machine learning algorithm. +An edit filter can define a rule to explicitly exclude particular malicious users from publishing, which cannot be straightforwardly implemented in a machine learning algorithm. %Fazit? @@ -149,7 +149,7 @@ Moreover, it is mostly the same old filters which have been highly active throug The list of the ten most active filters for each year since the introduction of the AbuseFilter extension is fairly stable. Although, as pointed out in section~\ref{sec:4-def}, filter patterns can be changed, they are mostly only optimised for efficiency, so it can be assumed filters have been catching the same troublesome behaviour over the years. Thirdly, the overall number of filter hits has risen since 2016 when a somewhat puzzling spike in filter hits which needs future investigation took place. -Additionally, the general tendency is that over time less good faith filter hits and more vandalism related ones occured. +Additionally, the general tendency is that over time less good faith filter hits and more vandalism related ones occurred. All in all, beside the peak in hits from 2016, additional temporal patterns of filters characteristics and activity can be explored. These include practices of filters' creation, configuration, and refactoring. @@ -221,7 +221,7 @@ Here, a comprehensive list of all these pointers for possible future research is \item \textbf{Is there a qualitative difference between the tasks/patterns of public and hidden filters?} According to the guidelines for filter creation, general filters should be public while filters targeting particular users should be hidden. Is there something more to be learnt from an examination of hidden filters' patterns? Do they actually conform to the guidelines? %One will have to request access to them for research purposes, sign an NDA, etc. \item \textbf{How are false positives handled?} Have filters been shut down regularly, because they matched more false positives than they had real value? Are there big amounts of false positives that corrupt the filters hit data and thus the interpretations offered by the current work? \item \textbf{To implement a bot or to implement a filter?} An ethnographic inquiry into if an editor is simultaneously an edit filter manager and a bot operator when faced with a new problem, how do they decide which mechanism to employ for the solution? - \item \textbf{What are the repercussions on affected editors?} An ethnographic study of the consequences of edit filters for editors whose edits are filtered. Do they experience frustration or allienation? Do they understand what is going on? Or do they experience for example edit filters' warnings as helpful and appreciate the hints they have been given and use them to improve their collaboration? + \item \textbf{What are the repercussions on affected editors?} An ethnographic study of the consequences of edit filters for editors whose edits are filtered. Do they experience frustration or alienation? Do they understand what is going on? Or do they experience for example edit filters' warnings as helpful and appreciate the hints they have been given and use them to improve their collaboration? \item \textbf{What are the differences between how filters are governed on EN Wikipedia compared to other language versions?} Different Wikipedia language versions each have a local community behind them. These communities vary, sometimes significantly, in their modes of organisation and values. It would be very insightful to explore disparities between filter governance and the types of filters implemented between different language versions. diff --git a/thesis/abstract_en.tex b/thesis/abstract_en.tex index b6286fc6892c0498bec7112ee37467325039335d..267d0a57a4a8466fb2a9b2da9a7c18271c3ea3ac 100644 --- a/thesis/abstract_en.tex +++ b/thesis/abstract_en.tex @@ -11,7 +11,7 @@ The present thesis offers an initial investigation of a previously unexplored by scientific research quality control mechanism of Wikipedia—edit filters. It is analysed how edit filters fit in the quality control system of English Wikipedia, why they were introduced, and what tasks they take over. -Moverover, it is discussed why rule based systems like these seem to be still popular today, when more advanced machine learning methods are available. +Moreover, it is discussed why rule based systems like these seem to be still popular today, when more advanced machine learning methods are available. The findings indicate that edit filters were implemented to take care of obvious but persistent types of vandalism, disallowing these from the start so that (human) resources can be used more efficiently elsewhere (i.e. for judging less obvious cases). In addition to disallowing such vandalism, edit filters appear to be applied in ambiguous situations where an edit is disruptive but the motivation of the editor is not clear. In such cases, the filters take an ``assume good faith'' approach and seek via warning messages to guide the disrupting editor towards transforming their contribution to a constructive one. diff --git a/thesis/conclusion.tex b/thesis/conclusion.tex index 12b6cad00412a319ceda492799f4a08600a4b84a..7773eb9a4dc5ca9823c44c93e1e5f580c20fd649 100644 --- a/thesis/conclusion.tex +++ b/thesis/conclusion.tex @@ -18,7 +18,7 @@ Additionally, interesting paths for future research were suggested. Summing up the most prominent results, edit filters, together with page protection and title/spam blacklist mechanisms, are the first mechanism verifying incoming contributions. By acting on unpublished edits they can disallow unconstructive ones directly and thus reduce the workload for other mechanisms. At the time of their introduction, the need was felt for a mechanism that swiftly prohibited obvious but difficult to remove vandalism, often caused by the same highly motivated malicious users. -Although mass-scale page moves to nonsenical names could be taken care of by admin bots, edit filters were viewed as a neater solution since this way such edits are not published at all. +Although mass-scale page moves to nonsensical names could be taken care of by admin bots, edit filters were viewed as a neater solution since this way such edits are not published at all. Also, with some dissatisfaction with bots' development processes (poorly tested and not available source code, low responsiveness of some bot operators), the opportunity for a clean start with a new tool was taken. Apart from targeting single highly motivated disrupting editors, edit filters take care of ``common newbie mistakes'' such as publishing text not formatted according to wikisyntax or erasing an entire page instead of properly moving it to a different name, or suggesting it to the formal Articles for Deletion process. By issuing warnings with helpful pointers towards possible alternative actions, edit filters allow a unintentionally disrupting editor to improve their contribution before re-submitting it. @@ -50,7 +50,7 @@ In a way, that is exactly what Wikipedia's edit filters are—they are triggered Since Wikipedia is distinctly relevant for the shaping of public opinion, despite its ``neutral point of view'' policy~\cite{Wikipedia:NeutralPointOfView} it is inherently political. At the beginnings of this research, I heard from a former colleague that there was an edit filter on the German Wikipedia targeting gendering. -``To gender'' is a linguistic praxis whereby words referring to people are explicitely marked to designate more genders than the standardly used generic masculine. +``To gender'' is a linguistic praxis whereby words referring to people are explicitly marked to designate more genders than the standardly used generic masculine. It is a political praxis aiming to uncover under-represented groups and their experiences through the conscious use of language. Even though no linguistic norm has established gendering to date, conscious decisions for or against the praxis are political, and so are technologies implementing these decisions. As it turned out, no such filter existed on the German Wikipedia diff --git a/thesis/introduction.tex b/thesis/introduction.tex index 050f4cfe92894ac9ca98edc2b41893c6767c305b..8763bde844ac4ca5618e103c474fdf758daccddb 100644 --- a/thesis/introduction.tex +++ b/thesis/introduction.tex @@ -10,14 +10,14 @@ \label{chap:introduction} In May 2014 the US American magasine \textit{The New Yorker} published a story called ``How a Raccoon Became an Aardvark'' in its column ``Annals of Technology''~\cite{Randall2014}. -It tells an anecdote about a student from New York who, some six years before, edited the Wikipedia article on ``coati'' (a member of the racoon family native to South and Central America) to state that the coati is ``also known as [...] Brazilian aardvark''~\cite{Wikipedia:Coati}. +It tells an anecdote about a student from New York who, some six years before, edited the Wikipedia article on ``coati'' (a member of the raccoon family native to South and Central America) to state that the coati is ``also known as [...] Brazilian aardvark''~\cite{Wikipedia:Coati}. This simple action is a mundane example of how Wikipedia works: Anyone can edit and small contribution by small contribution the world's largest knowledge base is created. Except, the student made the whole thing up and published on Wikipedia an inside joke he had with his brother on their holiday trip to Brazil. Unsourced pieces of information are not supposed to survive long on Wikipedia and he thought that the edit would be swiftly deleted. Fast-forward to 2014, not only had this part of the ``coati'' entry not changed, but it cited a 2010 article by the newspaper the \textit{Telegraph} as evidence~\cite{Wikipedia:CoatiEvidence}. -In the meantime, apparently several newspapers, a YouTube video and a book published by the University of Chicago~\cite{Henderson2013} claimed that the coati was known as Brazilan aardvark. +In the meantime, apparently several newspapers, a YouTube video and a book published by the University of Chicago~\cite{Henderson2013} claimed that the coati was known as Brazilian aardvark. It proved not trivial to erase the snippet from Wikipedia since there were all these other sources affirming the statement. By then, it was not exactly false either: The coati \emph{was} known as ``Brazilian aardvark'', at least on the Internet. @@ -152,7 +152,7 @@ The community was willing to disallow this kind of edits from the very start, re In addition to disallowing such vandalism, edit filters appear to be applied in ambiguous situations where an edit in itself is disruptive but the motivation of the editor is not clear. For example, deleting the entire content of a page could be malignant, but it could also be the result of a new editor not familiar with proper procedures for deleting or moving pages. In such cases, the filters take an ``assume good faith'' approach and seek via warning messages to guide the disrupting editor towards transforming their contribution into a constructive one: -In the page blanking example, a warning contains links to the documentation for redirects and the Articles for Deletion process~\cite{Wikipedia:ArticlesForDeletion}, and advices the editor to revert the page to the last uncompromised version in case it has been vandalised, and to use the sandbox for test edits. +In the page blanking example, a warning contains links to the documentation for redirects and the Articles for Deletion process~\cite{Wikipedia:ArticlesForDeletion}, and advises the editor to revert the page to the last uncompromised version in case it has been vandalised, and to use the sandbox for test edits. There are also a smaller number of filters which take care of various maintenance tasks—above all tracking a certain bug or other behaviour for further investigation. Since the current work is just a first exploration into edit filters, at the end, a comprehensive list of open questions for future research is compiled. diff --git a/thesis/thesis_main.tex b/thesis/thesis_main.tex index d1299b5aca6e9509e87dd7cff593b7ac0c7f44f8..948d493153863d08a9c7554fb2d6aa11fa72aefe 100755 --- a/thesis/thesis_main.tex +++ b/thesis/thesis_main.tex @@ -24,7 +24,7 @@ % \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} -\usepackage[english, ngerman]{babel} +\usepackage[english]{babel} \usepackage{ae} \usepackage{bibgerm}