Download pdf
Empirical Study on the Regulation of Data Crawling Behavior under the Anti-Unfair Competition Law
- 1 BeiJing Jiaotong University
* Author to whom correspondence should be addressed.
Abstract
Data crawling refers to the automated process of acquiring and storing web information, with data crawlers being one of its most widely used forms. The “webpage acquisition”—”webpage filtering”—”webpage storage” method of crawling, along with data transactions, often involves breaches of contract, infringements, unfair competition disputes, and other compliance-related legal risks. When courts handle such cases, they have generally adopted the Anti-Unfair Competition Law as the legal basis for regulating data crawling and its subsequent applications, achieving widespread legal consensus. In assessing the scope of unfair competition behavior, the judicial community has widely accepted a moderate extension of the criteria for identifying competitive relationships, and more cases are being adjudicated under the second article of the Anti-Unfair Competition Law. Courts generally use the Anti-Unfair Competition Law as the legal framework when reviewing data crawling behavior, while also emphasizing the balancing of multiple interests. At the same time, challenges arise in case rulings regarding the identification of competitive relationships, damage caused by competition, and the determination of business ethics, which necessitate the optimization of existing criteria and the introduction of new standards to enhance their recognizability and operability. Furthermore, when applying the general provisions of the Anti-Unfair Competition Law, judicial difficulties arise, calling for a return to the competitive law nature, achieving a regulatory model under dynamic competition, and introducing economic analysis standards to enhance the predictability of business ethics judgments.
Keywords
Data Crawling, Web Crawlers, Interest Balancing, Article 2 of the Anti-Unfair Competition Law
[1]. Data source from China Judgments Online (https://wenshu.court.gov.cn/) and Faxin official website (http://www.faxin.cn/). (2024, December 7).
[2]. Li, H., & Sun, J. (2018). On the legal boundaries of web scraping data behavior. Electronic Intellectual Property, 12.
[3]. Ding, X. (2019). Who owns the data? — A look at platform data ownership and data protection from the perspective of web crawlers. Journal of East China University of Political Science and Law, 5.
[4]. Beijing Weimeng Chuangke Network Technology Co., Ltd. v. Yunzhilian Network Technology (Beijing) Co., Ltd. (2017). Civil judgment (2017) Jing 0108 Minchu 24512, Beijing Haidian District People’s Court.
[5]. A certain computer system company, a certain technology (Shenzhen) company v. a certain technology company (Hangzhou) (2021). Civil judgment (2021) Zhe 8601 Minchu 309, Hangzhou Railway Transport Court.
[6]. Beijing Weimeng Chuangke Network Technology Co., Ltd. v. Shanghai Fuyu Culture Communication Co., Ltd. (2017). Civil judgment (2017) Jing 0108 Minchu 24510, Beijing Haidian District People’s Court.
[7]. Jiang, G. (2019). Reflection and interpretation of the Anti-Unfair Competition Law’s network provisions: Focusing on the principle of typification. Chinese and Foreign Legal Studies, 1.
[8]. Hangzhou Alibaba Advertising Co., Ltd., Alibaba (China) Network Technology Co., Ltd., et al. v. Nanjing Mazhu Network Technology Co., Ltd., et al. (2019). Civil judgment (2019) Zhe 0108 Minchu 5049, Binjiang District People’s Court, Hangzhou, Zhejiang Province.
[9]. Yang, H., & Qu, S. (2013). On the legal nature of web scraper agreements. Application of Law, 4.
[10]. Feng, B., & Zhang, J. (2023). Economic logic and judicial connection of Antitrust Law and Anti-Unfair Competition Law in the digital platform field. Legal Studies, 3.
[11]. Shenzhen Tencent Computer Systems Co., Ltd., Tencent Technology (Shenzhen) Co., Ltd., et al. v. Zhejiang Soudao Network Technology Co., Ltd., et al. (2019). Civil judgment No. 1987, Zhe 8601 Minchu, Hangzhou Railway Transport Court.
[12]. Zhang, S. (2023). On the Anti-Unfair Competition Law protection of data sets. Journal of Henan University of Economics and Law, 5.
[13]. Li, Z. (2021). Reflections and revisions on the regulation of data crawling behavior from the perspective of Anti-Unfair Competition Law. Journal of Jinan University, 6.
[14]. Shen, G., & Liu, Y. (2021). The regulatory dilemma and solutions for data crawling behavior under the Anti-Unfair Competition Law. China Circulation Economy, 1.
[15]. Gao, J. (2024). Legality of data crawling behavior from the perspective of competition law. Journal of Jiangxi University of Finance and Economics, 1.
[16]. Li, J. (2024). Judging the legitimacy of data crawling behavior: Reflecting on the substantial substitution rule. Journal of Political and Legal Studies, 2.
[17]. Yang, H. (2014). The impact of crawler agreements on internet competition relationships. Intellectual Property, 1.
[18]. Fan, C. (2015). Industry practices and unfair competition. Jurist, 5.
[19]. Yao, J. (2019). Guidelines for the use of corporate data. Tsinghua Law Review, 3.
[20]. Chen, G. (2023). A theoretical critique and rule improvement on the expansion of general provisions under the Anti-Unfair Competition Law. Legal Studies, 1.
[21]. Lin, L., & Zhang, Z. (2003). On the hierarchy of rights in conflicts of rights: An analysis from the perspective of normative jurisprudence. Journal of Zhejiang University (Humanities and Social Sciences Edition), 5.
[22]. Sollen, L. (2010). Legal lexicon: A toolbox for law school students (Wang, L. H., Trans.). China University of Political Science and Law Press. (Original work published 2010).
[23]. Cai, C. (2021). The regulation of data crawling behavior in competition law. Comparative Law Studies, 4.
[24]. Li, J. (2024). Judging the legitimacy of data crawling behavior. Journal of Political and Legal Studies, 2.
[25]. Xu, K. (2021). The legitimacy and boundaries of data scraping. China Law Studies, 2.
[26]. Zhang, P. (2013). The general provisions of the Anti-Unfair Competition Law and its application: Reflections on crawler agreements in search engines. Application of Law, 3.
Cite this article
Chen,J. (2025). Empirical Study on the Regulation of Data Crawling Behavior under the Anti-Unfair Competition Law. Advances in Social Behavior Research,16(2),1-11.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Journal：Advances in Social Behavior Research
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).