How Orwellian Is Chinas Social Credit System?

Lesen Sie die deutsche Version hier.

The end of 2020 also marks the end of the first phase in China’s Social Credit System (SCS). Since the publication of comprehensive plans for its construction in 2014, it has become the posterchild example of the increasingly authoritarian bent of the Xi administration, as well as the dystopian or “Orwellian” purposes to which digital technologies and data can be put. The reality has, however, always been more prosaic. China’s leadership is now embarking on the second, consolidated phase of SCS development. It is therefore worthwhile to review how SCS authorities have perceived the results of the past period, how it assesses successes and shortcomings, and towards which direction they will steer the SCS.

The notion of social credit emerged around the turn of the century, in response to a number of perceived ills. First and foremost, the new market economy was beset by rule-breaking where a relatively weak state could not provide an effective deterrent. While China has a reputation of strict authoritarian governance and centralized control, during the 1990s, the administration was fragmented, lacking material resources and expertise. Local governments, for instance, did not communicate effectively, meaning that individuals convicted for something in one town could move to the next and start all over. During that time, food scandals, intellectual property infringement and other economic wrongdoings were rife. This resulted in a deep deficit of trust. Social credit would remedy this through the comprehensive integration of government information, and imposing stronger sanctions against dishonest and non-compliant conduct. In other words, the SCS would serve as an amplification device for the enforcement of existing laws and regulations. Later on, information was also seen as useful in building up a consumer credit industry. China’s population had been largely unbanked, relying on hard cash for most transactions. Bereft of previous financial information, building up a consumer credit industry therefore required other data to assess creditworthiness. Finally, with the 2014 Plan, credit mechanisms were also proposed for internal government oversight.

It took, however, over a decade for the central government to move from conceptualisation to a comprehensive plan¹. As so often in Chinese policy, local governments moved first. As a result, a great variety of local social credit systems emerged. Some of them included quantitative scoring methods, others didn’t. Most of them were based on existing government information, but in some cases, local officials sought to use the SCS to ban behaviour they saw as undesirable, rather than unlawful. One mayor attempted to sanction frequent job-hopping², for instance. Action at the central level consisted mostly of binary blacklist systems, where individuals or businesses found guilty or particular infractions with designated aggravating circumstances, would face a range of expanded punishments. The most well-known blacklist punishes non-implementation of court verdicts, and includes, amongst others, a prohibition on air travel and luxury purchases, as well as ineligibility for certain jobs. The SCS also had a reputational element: much of the information in it is available publicly on creditchina.gov.cn, enabling due diligence of potential employees, borrowers or business partners. In some locations, telecommunications operators even cooperated with government authorities so that those receiving a call from a blacklisted number would receive notification to that effect.³

On the financial front, the People’s Bank of China allowed⁴ eight private businesses to experiment with credit rating models in order to develop China’s credit scoring industry. Giant platform operator Alibaba developed Sesame Credit⁵, a quantified scoring mechanism combining elements of a loyalty scheme with the sort of user assessment system present on many Western websites. Using big data and algorithmic scoring, Sesame Credit was far more sophisticated than the crude governmental systems, none of which involve private sector data or AI-based ratings. They became confused in media reports, however, leading to perceptions that the Alibaba model was a pilot scheme for governmental ambitions.

A policy document from late 2020⁶, however counters this. This document outlines the lessons Beijing has derived from the project’s construction thus far, and how it wishes to continue it. First, the number of blacklists should be controlled. Facing criticisms that the SCS was growing out of hand, the central government is cracking down on the establishment of new blacklists and has ordered a review of existing ones. Blacklists should only be based on central laws, regulations and policies, all others will be abolished. For Chinas private companies it has always been forbidden to put consumers on blacklists. Blacklists were and are state-run only. Second, considerable attention is now being directed to notification, appeal and credit recovery options. In a number of well-publicized early cases, individuals or businesses were not informed they had been blacklisted, and it was often unclear how one could leave a blacklist. These measures will be remedied, with a new policy to encourage credit recovery as soon as possible. Third, there are greater protections on which information can be entered into the SCS, with a core catalogue at a national level. Local governments can formulate supplementary catalogues, but these must still be based on personal information. Fourth, the new policy requires greater protection of personal and corporate data, and greater restrictions on which data will be made public.

In the area of financial credit, none of the eight businesses allowed to run pilot programmes, was allowed a permanent licence to issue credit scores. Instead, the People’s Bank of China (PBoC) established a credit scoring company, Baihang Credit, with participation from all eight companies, evading conflict-of-interest issues and perceived shortcomings in all systems. However, reports suggest large online platforms Tencent and Alibaba have been reluctant to submit their data to this company. To a certain degree, recent political scrutiny over these companies could be seen as an attempt to counter their data monopoly in the financial area.

The SCS never was one integrated, centralized system. Rather, it was an ecosystem with hundreds of discrete components, sharing the central notion that the conduct of individuals and businesses should be rewarded or punished and information could help in doing so. In the years going forward, the consolidation of the SCS will mean greater centralization of a hitherto highly diverse phenomenon, with stricter limits on what individual departments and local governments can and cannot do. An upcoming Social Credit Law, likely to emerge within the next years, will confirm these trends. The SCS may also play an increasingly useful role in fostering greater transparency over corporate conduct in China, something the government still seems to encourage.

So how Orwellian is the SCS? On the one hand, it is clear that it fits into the current Chinese leadership’s agenda of greater centralization and stricter control over individual and corporate activities. On the other hand, developments over the past year suggest that the leadership has grown far more aware about concerns about its functioning among the population, as well as the inevitable fact about government information systems: the more a system is expected to do, the more its performance of any given task degrades. The SCS is now more than ever restricted to being an amplification device for existing laws and regulations in cases of particular gravity, rather than a locus for decision-making itself. As such, it mirrors the broader development of China’s legal system. For instance, facial recognition camera detection enables blacklisting someone after jaywalking five times (as happens in Shenzhen). However, the SCS doesn’t cause the cameras to be there, it merely adds consequences. There are many concerns one could have about China’s use of technology and data in social management and surveillance. The SCS is by far not the most important or salient among them.

Is this likely to change in the future, for instance because of the introduction of big data or artificial intelligence? Obviously, anyone who claims to be able to predict how China is going to look ten or twenty years from now is either a liar or a fool. There are, however, a few trends one can identify. First and foremost, it is undoubtedly the case that Chinese security services are increasingly using smart digital tools to augment their surveillance capabilities. Moreover, the leadership has announced its intention to use digital technologies more broadly in enhancing governance, ranging from smart cities to long-distance education and healthcare. However, the SCS currently is, and is likely to remain, low in technology. Partly, this is due to the decision to limit SCS data to government-held data. It is also the case that the SCS does not make decisions itself – it merely reflects decisions taken elsewhere in the system. But another element is that Chinese authorities see technology as a way to augment, not replace, human decisionmaking. Self-learning algorithms, they fear, may well become uncontrollable, eroding the very authority the Party sees as essential to continued political stability. Similarly, there is now a wave of regulation targeting the use of algorithms in the private sector, and imposing greater accountability and liability. China’s government is  a great supporter of technology, but not an uncritical one.

A last question that often comes up is whether the SCS will spread outside China’s borders. To this, the answer is simple: not if the governments concerned do not want it. At a point in time where big technology companies seem insurmountable, and digital applications have become unavoidable elements of daily life, it is often forgotten that they can, and perhaps should, be regulated. There is no reason to believe the Chinese government will take active efforts to export the SCS. The decision, therefore, to what degree SCS-type systems emerge elsewhere around the world must be a matter of domestic political responsibility. And in Europe, citizens all have a voice in that judgment.