How Palantir works

One of the most mysterious companies of our time. Secret software developer for special services. A company that knows everything about you. So various media called the American software company Palantir Technologies, which works with data analysis.

Her clients include the army, police, intelligence agencies, banks, and more recently the US Department of Health, which commissioned Palantir to work on a platform to fight coronavirus.

How the company earned the dislike of human rights activists, what services it provides to the American security forces, how the Russian hacker's attacks on PayPal influenced its creation and why it is accused of facilitating the tracking of immigrants, ForkLog figured out.

Key

Palantir provides its products to a wide range of customers, but its main customers are intelligence agencies and government organizations.

Palantir software allows you to compare data from many sources, find relationships between them, and visualize them in the form of various charts.

Human rights activists believe that Palantir's cooperation with some US government agencies may violate human rights.

Founded in 2003, Palantir takes its name from the palantir stones in J.R.R. Tolkien's legendarium. With their help, it was possible to see what was happening or what happened in another place.

Palantir is the brainchild of renowned entrepreneur and investor Peter Thiel, so the Lord of the Rings reference is not surprising. He often draws inspiration from the Tolkien universe - the names of at least five companies, which Thiel co-founded, are taken from the works of Tolkien.

Thiel is known as "the PayPal mafia don". It was while working at PayPal that he got the idea to create Palantir. And the Russian hacker contributed to this.

In his book From Zero to One. How to create a startup that will change the future Thiel spoke about the problem that PayPal faced in the mid-2000s - the company could not track a large number of transfers and due to credit card fraud, it was losing more than $ 10 million monthly.

Initially, an automatic algorithm for tracking transactions, created by mathematicians, did not work - the scammers quickly adjusted to it and changed their tactics. Later, PayPal rewrote the program using a hybrid approach - "the computer marked the most suspicious transactions in the user interface, and the human operators already made the final decision whether it was fraudulent or not."

“We named this hybrid system Igor, after a Russian internet scammer who boasted that we could never deal with him,” Thiel writes.

Thanks to the new solution, the first quarter of 2002 was the first profitable quarter in PayPal history. The FBI got interested in the decision and asked permission to use "Igor" for solving financial crimes.

After the sale of PayPal, Thiel continued to think about a solution and shared with fellow student Alex Karp the idea of ​​a new startup - a symbiosis of humans and algorithms in data analysis to find terrorist networks and financial fraudsters.

Subsequently, this idea will be embodied in the creation of a software company that allows you to extract the most information from disparate data sources. She will be named Palantir, and Karp will take over as CEO.

***

In addition to Karp, Thiel recruited ex-PayPal engineer Nathan Gettings and two Stanford alumni Joe Lonsdale and Stephen Cohen to create Palantir. They had to develop an algorithm.

Despite the FBI's interest in the Igor solution, government agencies did not immediately notice Palantir, which offered a similar algorithm with an easy-to-use interface.

Initially, the development was fully funded by Thiel himself. Alex Karp, who was tasked with finding funding, was unlucky - dozens of investors turned down the startup.

However, soon Palantir was still interested - the company received $ 2 million from the CIA venture capital unit In-Q-Tel.

“The In-Q-Tel investment gave Palantir something more important than money: CIA approval. As the doors began to open in Washington, Palantir began to attract fans from intelligence and national security organizations, ”writes Intelligencer.

Subsequently, Palantir's clients included various government agencies, including the FBI, the US Department of Homeland Security, the NSA, the US Air Force and others.

Palantir's customers included the Los Angeles Police Department.

Sarah Brayne, a sociology professor at the University of Texas at Austin, has spent over two years studying how Los Angeles cops use the system. The results of her research not only lifted the curtain on how Palantir works, but also made many worry about the consequences.

The Palantir database contained standard information about citizens - name, gender, place of study, and the like. Additional information came into the system from government agencies, for example, from the US Department of Motor Vehicles. It was also populated with data purchased from private companies.

The system included not only offenders, but also their contact persons.

“I would warn against the thought that if you have nothing to hide, you have nothing to fear. This logic is based on the assumption that information [into the system] is entered without error or prejudice, ”Brain said.

Palantir software looks at a variety of data sources - financial documents, flight booking information, mobile phone data, social media messages - and looks for connections that analysts might miss, Bloomberg reported. It then presents the discovered connections in easy-to-interpret graphics that look like spider webs.

Reporters reviewed the user manual for the Palantir Gotham app used by law enforcement agencies.

According to him, the police could obtain extensive information about a person, practically having no data about him - for example, knowing only the name or number of the car.

Some data sources - such as marriage, divorce, birth, and business records - also included data on other people who are associated with the person. Sarah Brain called this secondary surveillance network.

“When the police become interested in someone, they don't just collect phone numbers, business relationships, the suspect's travels and the like. She also collects information about people associated with him, ”writes Vice.

A special tool allowed the police to filter, sort, display, analyze and export various data. Found relationships are visualized using charts, histograms or timelines - the tool selected the best option based on the analyzed data.

When a person appeared in the system, a police officer could "subscribe" to him, it follows from Brain's study of the interaction of law enforcement in Los Angeles with Palantir.

For example, a police officer could receive a notification on the phone when the car of a person who interested him was in a “suspicious” area. This was achieved by integrating data from outdoor video surveillance cameras into the Palantir system.

“Without public scrutiny or regulation, Palantir helped the LAPD build a vast database of names, addresses, phone numbers, license plates, friendships and romantic relationships — guilty, innocent, and those in between,” writes BuzzFeed.

Another customer for Palantir was the US Immigration and Customs Enforcement (ICE). In 2014, she awarded the firm a $ 41 million contract to build and maintain a tracking system.

Human rights activists are convinced that Palantir played a key role in the deportations of illegal migrants. The company insisted that they did not work with the ICE department responsible for the detention of undocumented immigrants and deportation.

However, Palantir's assurances have not convinced human rights defenders. Amnesty International criticized the company's contracts with ICE and said Palantir was "failing to meet its human rights obligations."

Particular attention to the collaboration between Palantir and ICE is also riveted because of Peter Thiel's ties with US President Donald Trump, who is known for his anti-immigrant rhetoric. Thiel is perhaps the only Silicon Valley representative to openly support Trump.

Karp, who harshly criticized Trump in 2015, was spotted meeting with him among the leaders of large technology companies immediately after the latter was elected president.

Palantir's work with ICE has sparked internal resentment among the company's employees. Karp said that cooperation with the authorities will continue.

"Silicon Valley says to the average American:" I'm not going to support your need for defense, "while it itself sells developments to countries hostile to America. This is the position of the losers, "Karp said.

Such patriotism is almost the fundamental idea of ​​Palantir. The company's filed to go public says it "does not work with clients or governments whose positions or actions are incompatible with the mission to support Western liberal democracy and its strategic allies."

The fact that the main clients of Palantir are precisely the US security forces and intelligence services fully reflects the company's commitment to this mission.

As Bloomberg wrote, according to some reports, the Palantir system even helped eliminate the terrorist Osama bin Laden. The company does not confirm or debunk these rumors. Thiel in his book only stated that "he cannot tell anything about that operation."

He noted that with the help of programs created by Palantir, analysts were able to predict "where the Afghan rebels are planning to set the bombings."

At the same time, Palantir is trying to ease its dependence on government contracts and expand its client base. Commercial clients work with Palantir Foundry, a data integration platform.

The financial holding JPMorgan became one of the first corporate clients of the company. However, this collaboration was initially clouded by a scandal.

In 2009, Palantir specialists worked with the bank's insider threat detection team, Bloomberg reports.

Palantir software analyzed e-mail, browser history, transcripts of telephone conversations, identifying keywords and behavior patterns of bank employees.

For example, the Palantir algorithm alerted JPMorgan's insider team when an employee started arriving late for work, a sign of potential frustration.

The workers were shocked that “no one from the bank or Palantir set any restrictions,” the newspaper writes.

This story ended with the bank's top managers finding out that they too were being followed.

"The platform developed for the war on terrorists has been used against ordinary Americans," Bloomberg sums up.

To meet its privacy and security commitments, Palantir catalogs and labels system users' data - everything is recorded.

The company is also trying to follow certain "ethical standards" - an ethics hotline called Batphone has been set up for Palantir employees. They can use it to anonymously report to Palantir management about work for a client that they consider unethical.

A hotline appeared after the WikiLeaks scandal. In 2011, it became known that one of the employees discussed in emails the possibility of spying on WikiLeaks informers in order to prevent the organization from disclosing documents from Bank of America.

Karp has made a public apology and said Palantir supports the right to free speech and privacy.

Karp himself believes that the company can find a balance between protecting privacy and ensuring security.

"I did not sign up for the government to know when I smoke a joint or [with whom] I am having an affair," Karp is quoted by Forbes.

In his letter, Karp almost directly accused Silicon Valley companies like Facebook and Google of selling personal data and intruding on privacy.

“Software projects with our country's defense and intelligence agencies, whose missions are to keep us safe, have become controversial, while companies based on advertising money have become commonplace,” Karp writes.

He noted that the Silicon Valley engineering elite may know more about software creation than anyone else, but "don't know more about how society should be organized and what justice requires."

But as Intelligencer points out, his argument ignores the fact that Palantir was used to analyze data from social media, including Facebook posts.

***

In 2020, Palantir directly listed its shares on the New York Stock Exchange (NYSE).

According to documents disclosed by the US Securities and Exchange Commission, the company had 125 clients in the first half of 2020. Palantir's revenue for this period reached $ 481.2 million.

It also follows from the documents that revenue from government contracts increased by 76%, and from commercial contracts - by only 27%. Cumulatively, since June 2019, it has almost halved.

At the same time, Palantir has suffered losses every year since its inception, the company itself pointed out in the section on risks. A significant portion of the revenue comes from a limited number of customers.

In addition, according to Bloomberg Businessweek, investors will have little or no influence on management decisions, since "Palantir is designed in such a way that Thiel, Karp and third co-founder Stephen Cohen control half of the voting shares through the trust indefinitely."

The publication notes that the company can afford it in part because its software is "almost perfect for solving the problems associated with the coronavirus pandemic."

In late April, the US Department of Health and Human Services (HHS), which is responsible for fighting the pandemic in the United States, awarded Palantir a $ 25 million contract. The company is to oversee the HHS Protect data collection and analysis platform.

“The deal was one of more than a hundred that Palantir struck in the early days of the pandemic - 83 of them between March and early April - and this is one of the reasons for the improvement in the company's financial performance this year,” Bloomberg Businessweek notes.

Against the background of this deal, several members of Congress at once asked what kind of data HHS Protect will collect and how they will use it, recalling the scandal with ICE.

“We have concerns about how the existing Palantir system used to track and arrest immigrants will be supplemented with personal health information from HHS Protect,” the letter said.

Deploying large-scale citizen tracking systems during a pandemic is a serious concern for human rights defenders. Especially when the companies creating such systems are closely connected with the authorities.

Back in 2011, Peter Thiel told Bloomberg that civil liberties advocates should welcome the creation of Palantir:

“We cannot afford another 9/11. That day “opened the doors” to all kinds of insane abuses and draconian politics. "

In his opinion, the best way to avoid such scenarios in the future is to provide the government with advanced technologies and ensure protection against their misuse.

However, after many revelations of government tracking of citizens, the world seems to no longer believe in the infallibility of intelligence officials or government agencies in matters of privacy.

Many years have passed since the revelations of Edward Snowden, and tracking systems continue to evolve and scale, as evidenced by numerous studies.

Aggregating the data collected by these systems in one database is fraught with the creation of a tool that literally knows everything about you. But where would there be room for privacy?