December 17, 2024 - Comments Off on Govt admits internet disruption due to “security” concerns
All Posts in
December 17, 2024 - Comments Off on More than 91% of Pakistan has access to cellular services: PTA
More than 91% of Pakistan has access to cellular services: PTA
December 17, 2024 - Comments Off on Limits on data scraping—Terrible news for social media research and platform accountability?
Limits on data scraping—Terrible news for social media research and platform accountability?
Early last year, following Elon Musk’s takeover, X (formerly Twitter) introduced paid API access tiers, dealing a deathblow to its long-standing role as a uniquely valuable resource for academic research.
APIs, or Application Programming Interfaces, act as bridges that connect two software applications, enabling data exchange. For instance, an API can be used to request data from X and store it in a structured format like a CSV file. Paid API tiers mean that the previous official Twitter/X API is no longer functional, which further means data cannot be scraped from X for free.
This measure, alongside introducing ‘rate limits’ (limits to the number of tweets users could view per day), was ostensibly taken to prevent what Musk referred to in a July 2023 tweet as “...extreme levels of data scraping & system manipulation…”, which were slowing down the site and hindering the user experience. However, no further explanation was provided by Musk to support the claim of “extreme levels of data scraping”. X users pointed out that if these indeed exist, they would likely manifest as anomalies that can be identified as request spikes, and thus easily narrowed down to identify and ban users requesting abnormally large amounts of data.
Unreasonably priced paid API tiers
Nevertheless, Musk’s crackdown on data scraping continued, and as of now there are four API tiers: free, basic, pro, and enterprise. The free tier allows 100 reads per month, a negligible amount for researchers and academics looking to analyse posts on X for meaningful insights, patterns, and key data points. The most affordable paid tier only offers around “0.3 percent of [data] [third parties] previously had free access to in a single day”, with enterprise tier prices shooting to upwards of $42,000 per month. It is also impossible to read posts without being logged into X.
X’s crackdowns on data scraping
As a means to enforce these measures, X has regularly been cracking down on third-party API access, blocking IP addresses and mass suspending developer accounts, and bringing costly lawsuits against non-profits whose research utilises data scraped from X without authorised access.While one might believe Musk’s policies aim to combat misuse and preserve platform integrity, the fact that X, a year after introducing paid API tiers, amended its terms of service to permit training its generative AI model on user posts without their consent makes such intentions harder to trust.
The value of data scraping to social media research
These recent and ongoing changes to X’s policies represent a strange and worrying landscape for data accessibility in academic research. Academic researchers have long relied on X’s open model and public data for producing valuable medical research, including enriching datasets for cancer treatment methods, and tracking mental health trends during public health emergencies like COVID-19.
In addition to X, scraped data across platforms has also been used to collect evidence for studies that have offered positive benefits to society, such as uncovering illegal markets for adopted children on Yahoo bulletin boards, revealing overlaps between law enforcement and extremist Facebook groups, and identifying problems with TikTok’s algorithm targeting youth. From the digital rights lens, this model has improved understanding of the implications of harmful online content for society and provided the rights community an opportunity to share policy recommendations. Without it, tracking social media companies’ adherence to their own content moderation rules for regulating harmful content is increasingly challenging.
To the dismay of researchers, X is not the only platform that restricts access to their API. Facebook, Instagram, and LinkedIn, have long restricted access to user data.
The legal debate around scraping public data
Legally, social media companies like X and LinkedIn have come under fire for bringing claims against the scraping of public data on their platforms. In X Corp. v. Bright Data Ltd. (2024), X Corp asserted breach-of-contract and tort claims against a data scraping company, Bright Data, to prevent it from extracting and copying public data from X, and selling tools that enable users to extract and copy public data from the same.
The case was ultimately dismissed because none of the claims passed muster, and in its ruling the U.S. district court quoted a previous judgement for a case brought by LinkedIn against data scraping by an airline website in 2022:
…giving social media companies ‘free rein to decide, on any basis, who can collect and use data - data that the companies do not own, that they otherwise make publicly available to viewers, and that the companies themselves collect and use - risks the possible creation of information monopolies that would disserve the public interest.’
HiQ Labs, Inc. v. LinkedIn Corp., 31 F.4th 1180, 1202 (9th Cir. 2022) qtd. in X Corp. v. Bright Data Ltd., C 23-03698 WHA, 2 (N.D. Cal. May. 9, 2024)
These recent court rulings highlight the risks posed to public interest by the arbitrary crackdown on data scraping. They underscore how social media platforms hold “arms length”, non-exclusive rights to user data, and have no written exclusive copyright license over user’s works. Users own sole exclusive rights to their data, and only owners of exclusive rights can seek protection and remedies from courts. Litigation brought by X against data scraping of public data is an overreach of its rights, as explained in the X Corp. v. Bright Data case judgement:
…invoking state contract and tort law, X Corp. would entrench its own private copyright system that rivals, even conflicts with, the actual copyright system enacted by Congress. X Corp. would yank into its private domain and hold for sale information open to all, exercising a copyright owner's right to exclude where it has no such right.
X Corp. v. Bright Data Ltd., C 23-03698 WHA, 2 (N.D. Cal. May. 9, 2024)
Tools that bypass data scraping restrictions
Irrespective of legal woes and crackdowns, developers have started building third-party tools, which allow users, including researchers, to ‘informally’ scrape data from social media platforms.
These tools avoid the hassle of requiring official API access from platforms by mimicking human browsing behavior to extract data instead of relying on APIs. Since many of these tools offer low-code and no-code solutions to data scraping, they expand the user base of a practice that was in the past only limited to programmers with at least an intermediate understanding of Python and other programming languages. Now the layperson or researcher also has a wealth of data at their fingertips. Additionally, in the case of X, some of these tools allow for access to historic data going back to 2006, which even the official X API does not offer. At the same time, their unofficial status leaves these tools at a constant risk of bans, and open to litigation, making them unreliable options for long-term research.
How platforms can address data scraping responsibly
As such, the most practical, reliable, and legal option would be for platforms to address their duty towards offering researchers above-board methodologies to continue their noble and essential aim of knowledge production. Blanket bans on data scraping are not a one-size-fits-all solution. It is necessary to consider context and intent of use for each instance of scraping. It is true that the open web runs the risk of being ‘spidered’ by massive web crawlers that not only require massive bandwidth and overload servers, but are also being used to train Large Language Models (LLMs) without user and website owner consent. This is and should be a genuine cause of concern for tech companies and other stakeholders. At the same time, as discussed in detail above, data scraping specific sites and pages has time and again proven to be useful, valuable, and in need of protection.
For some skeptics, the problem then is not data scraping, but platforms evading accountability when it comes to contextualising use cases for data scraping. Legitimate, positive use cases can be identified by looking at research objectives, the size of data collection, and the use of appropriate safeguards to protect data and sites. By facilitating researchers with authorised free access for these legitimate use cases, platforms can reduce unofficial scraping on their sites, while also upholding their duties towards the public good. Putting significant amounts of data behind unbreachable paywalls, regardless of its intended use, represents a fundamental failure of platform accountability in supporting essential research and knowledge production.
By Sara Imran, Research Associate, Digital Rights Foundation
December 16, 2024 - Comments Off on No plans to block VPNs despite technical capacity, PTA Chairman
No plans to block VPNs despite technical capacity, PTA Chairman
December 16, 2024 - Comments Off on Bill to centralise citizen’s data tabled in the National Assembly
Bill to centralise citizen’s data tabled in the National Assembly
December 16, 2024 - Comments Off on November 2024 Newsletter: DRF collaborates with CSOs to set up Global Majority House in Brussels
November 2024 Newsletter: DRF collaborates with CSOs to set up Global Majority House in Brussels
DRF—in collaboration with the Center for Studies in Freedom of Expression and Access to Information (CELE), 7amleh, WHAT TO FIX, The London Story Foundation, Citizens’ Association (CA) “Zašto ne (Why not)”, and Myanmar Internet Project—set up the Global Majority House on 15 November to drive critical dialogue and collaboration between the Global South and Europe on digital governance. According to DRF Executive Director Nighat Dad, “We hope to contribute to conversations on systemic risks and crisis protocols while engaging with the broader ‘Brussels effect’ to shape global standards that resonate with the values and realities of the Global Majority.” Read more here.
Policy Initiatives:
Meeting with President Macron on tech and human rights
DRF’s Executive Director Nighat Dad participated in the Paris Peace Forum as part of the tech leaders cohort. The Paris Peace Forum is a platform that puts global governance, rule of law and peace on the international agenda. Ms Dad met with President Emmanuel Macron and other industry leaders to discuss the human rights implications of emerging technologies like AI, and challenges to digital governance.
Asia-Pacific Civil Society Consultation Report for Beijing+30
The Beijing+30 Asia-Pacific Civil Society Steering Committee, of which DRF is part, released a report titled Forging Gender-Just Futures ahead of the Beijing+30 proceedings in November. This report highlights priority areas of action to advance women’s human rights and gender equality based on the lived experiences of civil society members in the region. Read the report here.
Campaign on International Day to End Impunity Against Journalists (2 November)
On International Day to End Impunity Against Journalists, DRF called attention to the lack of implementation of the Protection of Journalists and Media Professionals Act, specifically regarding the non-establishment of a Commission for investigating, prosecuting and penalising rights violations against journalists. See our explainer here.
Campaign on International Day for Tolerance (16 November)
To mark International Day for Tolerance, DRF reshared the story of Laila, a trans femme TikTok influencer who experiences a unique form of online gender-based violence. See the English version here, and the Urdu version here.
Campaign on 16 Days of Activism Against Gender-Based Violence (25 November to 10 December)
DRF kicked off its campaign on 16 Days of Activism Against Gender-Based Violence with ‘Stay Cybersafe’ daily tips. Women and girls also shared their vision of what Pakistan would look like free from gender-based violence, a fitting reminder on International Day to End Violence Against Women (25 November). On 29 November, DRF marked International Women Human Rights Defenders Day by saluting women activists who raise their voices to help others, and raising awareness about psychosocial support for their emotional wellbeing.
View this post on Instagram
Blogpost on Platform Accountability
As part of DRF's new series on Platform Accountability, DRF analyzed a case of online gendered violence against trans community member Dolphin Ayan Khan, in which a video containing sensitive images of her was posted non-consensually on Facebook, X and YouTube. Did these platforms act to take the video down, or suspend accounts posting malicious web links that claimed to offer full access to the video? Read DRF’s analysis here.
Press Coverage:
CII decree against VPN usage criticized
Along with other rights activists, DRF's Executive Director Nighat Dad responded to the Council of Islamic Ideology (CII) decree against VPN use. She pointed out that the move to block VPNs conflicts with our fundamental right to privacy, and aims "only to target certain social media users". Read more here.
A study on AI generation of hyper-realistic images
Commenting on a study on AI, DRF's Executive Director Nighat Dad highlighted inclusivity issues and the potential misuse of hyper-realistic AI images. She said, “Including local skin tones within a predominantly white dataset is an essential step toward building inclusive AI models.” She also expressed concern about AI's accelerated progress, particularly in Pakistan where such technology is still a new phenomenon and has been used for exploitation and scams. Read more here.
DRF was also cited in the following press coverage:
Events:
DRF participates in this year's Beijing+30 review
DRF participated in the final review of the Beijing Platform for Action (BPfA), of which Pakistan is signatory, known as the Beijing+30 review. As a member of the Beijing+30 Asia-Pacific Civil Society Steering Committee, DRF was honoured to work together with civil society groups around the region to advance Beijing+30 gender equality goals.
DRF representative Irum Shujah, participated in the Young Feminist Forum on 16 November, where over 135 young feminist leaders from the Asia-Pacific shaped key demands for the Beijing+30 review. Their collective statement highlighted systemic oppressions rooted in colonialism, patriarchy, and racism, calling for transformative, intersectional, and inclusive feminist actions to reclaim rights and address the lived realities of youth in diverse contexts.
Ms Shujah also attended the preparatory CSO Forum held on 17-18 November, which brought together diverse voices from across the region to drive civil society engagement in the process.
Finally, Ms Shujah participated in the Ministerial Conference on the Beijing+30 Review, held on 19-21 November. This high-level forum, organized by the Asia Pacific Forum on Women in collaboration with UN Women, convened ministers, officials, and key stakeholders to evaluate the region's progress under the BPfA, address legal and policy challenges, and strengthen commitments to gender equality and women's empowerment.
Digital Literacy & Citizenship Bootcamp in Bagh, Azad Kashmir
DRF conducted its fifth Digital Literacy and Citizenship Bootcamp for 58 students at the Women University in Bagh, Azad Jammu and Kashmir, with the support of the U.S. Embassy, Islamabad. The students learnt about cyber harassment, online safety safety, disinformation and misinformation in today’s digital ecosystem, as well as the emergence of AI and what tools they can use to optimize the use of digital technology.
Other events:
- As part of the Beijing+30 process, the DRF legal team attended an online info session titled Road to Beijing+30 and Beijing 101 organized by the Asia Pacific Forum on Women, Law, and Development (APWLD) on 5 November.
- The DRF legal team participated in a UNESCO webinar titled "AI in the Courts: Transforming Justice with Practical Applications and Use Cases" on 21 November. The webinar brought together representatives from various countries to share insights on integrating AI into judicial processes. The discussion highlighted innovative approaches to leveraging AI for enhancing judicial efficiency, while emphasizing the importance of safeguarding justice, protecting human rights, and ensuring ethical, responsible implementation.
- DRF took part in the iMap’s Community Of Practice, a regular gathering of those who are affected by network interference and where regular updates are discussed among iMap’s community of civil society organizations.
Tech trends and tips
New tech and platform developments:
Unregistered VPNs are being blocked in Pakistan
The Pakistan Telecommunication Authority (PTA) has started its second trial to block unregistered VPNs ahead of a nationwide crackdown starting December 1, citing security risks and misuse concerns. Over 25,000 VPNs have been registered so far, with the PTA streamlining the process to ensure compliance.
Tip of the month:
Every time you post on social media, send an email or do a Google search, you leave a trace. These traces are actually permanent data trails which are tracked, analyzed and used to construct your 'profile'. Your profile is then used by businesses to bombard you with offers and adverts. This trail, consisting of information you leave and activities you do on the Internet, is called digital footprint.
You can reduce your digital footprint by:
- Self-Doxxing: Google yourself first and then contact the relevant sites to take down your personal information
- Setting up Google alerts: Set up a Google alert to keep yourself updated if your name is being mentioned somewhere on internet
- Deactivation of unwanted accounts: Delete or deactivate any of your accounts which are no longer required otherwise your profile will still be visible to others
- Privacy Policy: Always read the terms and condition before subscribing to online services
- Don't be fooled by surveys: Never fill out any survey that pops-up while browsing because someone is collecting your data
- Clearing cookies and caches: Always make sure to clear cookies and remove caches to prevent site tracking
DRF Resources:
Cyber Harassment Helpline:
The Cyber Harassment Helpline received 186 complaints in total in November 2024, around 80% of which were related to cyber harassment.
If you’re encountering a problem online, you can reach out to our helpline at 0800-39393, email us at helpdesk@digitalrightsfoundation.pk or reach out to us on our social media accounts. We’re available for assistance from 9 am to 5 pm, Monday to Sunday.
IWF Portal
DRF in collaboration with Internet Watch Foundation (IWF) and the Global Fund to End Violence Against Children launched a portal to combat children’s online safety in Pakistan. The new portal allows internet users in Pakistan to anonymously report child sexual abuse material in three different languages- English, Urdu, and Pashto.
StopNCII.org
Meta along with Revenge Porn Helpline (RPH) has launched a portal to support victims of Non-Consensual Intimate Image Abuse (NCII). NCII is a free portal for reporting cases of sensitive or sexual content existing online. Once you report a case, the necessary steps will be taken to block the images from the platform.
December 15, 2024 - Comments Off on Govt expected to push amendments to expand PECA powers this week
Govt expected to push amendments to expand PECA powers this week
December 14, 2024 - Comments Off on Journalists, vloggers among 150 booked under PECA for “maligning security agencies”
Journalists, vloggers among 150 booked under PECA for “maligning security agencies”
December 13, 2024 - Comments Off on Senate passes Digital Forensics Agency Bill
Senate passes Digital Forensics Agency Bill
December 13, 2024 - Comments Off on Only 38,000 freelancers have bank accounts in Pakistan: Government