LinkedIn Explained

By: Linda Bardha, Dominique Haywood, Yajing Hu

How the algorithms work on the LinkedIn Platform?

LinkedIn Feed Algorithms

People you may know

  • People you may know (started in 2006, began with a python script)
  • LinkedIn pre-computes the data by recording 120 billion relationships per day in a Hadoop     MapReduce (It runs 82 jobs which require 16TB of data) 
  • Apache Hadoop is a collection of open-source software utilities that facilitate using a network of many computers to solve problems involving massive amounts of data and computation. It provides a software framework for distributed storage and processing of big data using the MapReduce programming model
  • There are 5 test algorithms continually running – producing approximately 700 GB of output data for the ‘People You May Know’ feature.

Skill endorsement

  • After a member endorses a certain skill, the recommendations are stored as a key-value, by mapping a member id to the list of other members, skills id’s and the score.
  • The output is then used by the front-end team to display in the profile of a member

Jobs you may be interested in

  • LinkedIn uses Machine Learning and Text Analysis algorithms to show relevant jobs to a member’s profile
  • 50% of LinkedIn engagement comes from “Jobs you may be interested in” feature
  • The textual content like skills, experience, and industry are extracted from a member’s profile. Similar features are extracted from the job listings available in LinkedIn.
  • A logistic regression model is applied to know about the ranking of relevant jobs for a particular LinkedIn member based on the extracted features.

How does LinkedIn use AI & What are the consequences?

 

  • For employees

 

      • LinkedIn uses artificial intelligence in ways that employees experience everyday, such as giving them the right job recommendation, encouraging them to connect with someone, or providing them with helpful content in the feed.
      • LinkedIn can extensively personalize the recommendations and search results for employees with deep learning. To perform personalization at their level, LinkedIn need machine learning algorithms that can understand content in a comprehensive fashion. LinkedIn also leverage deep learning to automatically learns complex hierarchical structures present in data using neural networks with multiple layers to understand content of all types.
      • AI systems can help LinkedIn find patterns from huge amounts of data. LinkedIn has a rich collection of data from many different sources. Without AI systems, this work can be a time-consuming process. If the employees are a passive candidate who’s not looking for a job, LinkedIn is careful to only surface jobs that are really good and help them get to the next opportunity. If they are an active candidate, LinkedIn sometimes takes more risk and show them the jobs that may or may not be in the ballpark. Using all the past data about LinkedIn members and what members look like, LinkedIn is able to teach machines what are the appropriate jobs for the members.
      • LinkedIn also works in the background, doing things like making sure that employees are protected from harmful content, routing connection to ensure a fast site speed experience, and making sure that the notifications sent to employees are informative, but not annoying.

 

  • For employers

 

    • LinkedIn can help employers source and manage candidates, and therefore save time. LinkedIn announced more than one million open candidates who have signaled that they are open to new opportunities. LinkedIn trains algorithms on the huge amounts of data with such signals, and then those algorithms can predict who might be the best fit.
    • LinkedIn also help the employers’ jobs reach the right people. By looking at deeper insights into the behavior of applicants on LinkedIn, it starts to predict not just who would apply to their jobs, but who would get hired. AI system can allow employers to select the exact qualifications in the candidate they are looking for. Employers can define specific skill-sets, years of experience, and levels of education associated with a particular job title for more precise targeting. Using machine learning, LinkedIn will only serve the job to the applicants who a good fit for the their role.
    • LinkedIn also explores how artificial intelligence can go beyond just their own products through integrations such as Recruiter System Connect. It is working closely with its partners to deliver the most robust integration with Applicant Tracking Systems (ATS). Companies that are turning on Recruiter System Connect are powering their “Past Applicants” spotlight, which guides them to the best candidates based on the interactions stored in their ATS, and are seeing half or more of all messages get responses.

Source of data input

    • LinkedIn’s approach to AI is neither completely machine-driven nor completely human-driven. It’s a combination of the two. Both elements working together in harmony is the best solution.
      • AI systems that LinkedIn are reliant on human input and automated process. Take the example of profile data. At a fundamental level, almost all the member data is generated by members themselves. This might lead to a problem for AI system. One company might have a job named “senior software engineer”, while another company might provide almost the same role but named “lead developer”, and there might be other names as well. Humans can easily understand that these names are the similar concept, while for computers, it can be a challenging task. Consequently, standardizing data in a way that AI systems can understand is an important first step of creating a good search experience, and that standardization involves both human and machine efforts.
      • LinkedIn has taxonomists who create taxonomies of titles and use machine learning models that then suggest ways that titles are related. Understanding these relationships allows LinkedIn to infer further skills for each member beyond what is listed on their profiles.

Consequence

    • LinkedIn’s AI systems have had a huge impact for employees who are trying to find a job. With the personalization recommendation, LinkedIn saw a 30% increase in job applications.
    • Job applications overall have grown more than 40% year-over-year, based on a variety of AI-driven optimizations that have been made to both sides of the member-recruiter ecosystem.
    • AI-driven improvements to employers’ products have helped increase InMail response rates by 45%, while at the same time cutting down on the notifications that LinkedIn sends to our members.
    • AI has improved article recommendations in the feed by 10-20% (based on click-through rate).

Ethical/ Societal Ramifications of AI and Hiring

  • AI’s presence in hiring is intended to streamline the resume review and candidate selection processes. AI in hiring is also designed to avoid gender, racial and other biases.
  • LinkedIn’s new AI tool, which the company briefed Business Insider on appears designed to filter out the biases in data that can taint AI technology.
  • LinkedIn will track what happens in the hiring process with regards to gender, showing companies reports and insights about how their job postings and InMail are performing on this.
  • LinkedIn will re-rank the top search results in LinkedIn Recruiter to be more representative of the candidate pool for the posted jobs.
  • This current feature is only scaled to manage gender diversity, not racial or other demographics
  • LinkedIn has shows how its data could be used to map a person’s career based on their personality traits and interests, changing the order in which candidates are highlighted in could have individual and industry wide ramifications.
  • The technology industry is consistently highlighted in the media for issues with gender however, the top three jobs recruited through linkedin are all in the technology sector (DevOps engineer, Enterprise account executive, Front end engineer)
  • This product combats Amazon’s recruitment tool which was designed to identify the top candidates for Amazon roles
    • This product was pulled in October 2018 because of a bias designed into the tool, which preferred male candidates over female candidates.

Positive Impacts of AI on Recruiting and Hiring

  • AI allows hiring companies to find a larger pool of candidates through sites like LinkedIn by providing access to potential employees that may have otherwise been overlooked
  • AI can help employers process resumes and eliminate some of the processing time that it takes to review resumes and interview candidates
  • Recruiting profiles of typical employees can be created and used to hire and diversify an existing team

Negative Impacts of AI on recruiting and Hiring

  • Racial/Gender bias can eliminate groups of candidates because of factors other than qualification for the job
  • Putting distance between the interviewer and the potential employee can negatively impact the new employee’s view of  the company
  • This could impact employee retention and job satisfaction

References:

DeZyre. How LinkedIn uses hadoop to leverage big data analytics. 2016. Found at https://www.dezyre.com/article/how-linkedin-uses-hadoop-to-leverage-big-data-analytics/229?fbclid=IwAR3Du5ubRs6hKHt-e-8PnrEkttZL8QkapxiJEK_X355GzbGmzZyk_VPxGxk

Boyd, Joshua Brandwatch. “The LinkedIn Algorithm:How it works” 2018 found at https://www.brandwatch.com/blog/linkedin-algorithm/

Cuofano, Gennaro. LinkedIn Feed Algorithm. Found at https://fourweekmba.com/linkedin-feed-algorithm/

Deepark, A. (2018). An Introduction to AI at LinkedIn. Retrieved from https://engineering.linkedin.com/blog/2018/10/an-introduction-to-ai-at-linkedin

Josh, J. (2017). How LinkedIn Uses Automation and AI to Power Recruiting Tools. Retrieved from https://business.linkedin.com/talent-solutions/blog/product-updates/2017/how-linkedin-uses-automation-and-ai-to-power-recruiting-tools

Rosalie Chan. LinkedIn is using AI to make recruiting diverse candidates a no-brainer. https://www.businessinsider.com/linkedin-new-ai-feature-increase-diversity-hiring-2018-10

Alison DeNisco Rayome  The 3 most recruited jobs ever on LinkedIn are all in tech https://www.techrepublic.com/article/the-3-most-recruited-jobs-ever-on-linkedin-are-all-in-tech/