New Study on Web Scraping of Real Estate Websites Shows Serious Cause for Concern
SAN FRANCISCO, California, Oct. 7, 2015 — Distil Networks, Inc., the global leader in bot detection and mitigation, and Clareity Consulting, the premier technical consulting firm for the real estate industry, today announced the, "2015 Study on Real Estate Website Scraping and MLS Data Security." The report examines two surveys that sought to understand the barriers to adoption of anti-scraping technology - one with Multiple Listing Service (MLS) executives, and another with the owners of companies that provide Internet Data Exchange (IDX) websites and Virtual Office Websites (VOW). "The real estate industry faces serious challenges with intellectual property infringement via web scraping, yet they've been slow to adopt effective anti-scraping tools," said Rami Essaid, co-founder and CEO of Distil Networks. "Scrapers use bots to grab online real estate content, like property listings, and then create derivative products such as lead gen programs and appraisal data." The study reveals the perceptions of MLS and IDX executives regarding web scraping and how each member of the MLS data supply chain can align to protect the integrity of real estate industry data. The Findings MLS Executives - One hundred MLS executives responded to the survey, representing over half a million MLS members, brokers, and agents. 99 percent of MLS executives say compliance with rules protecting misuse of MLS data is important and 95 percent agree that IDX sites should be subject to rules specifically mandating anti-scraping protections. 98 percent of respondents want a set of standardized tests to facilitate anti-scraping compliance reviews, yet no test currently exists. 94 percent of MLS executives indicate a vendor's information security practices and sophistication of anti-scraping technology are important to them when selecting a technology vendor. IDX/VOW Vendors - 14 vendors representing over 400,000 websites were surveyed. IDX and VOW vendors operate on behalf of the MLS to serve agent sites and other websites that display and syndicate MLS listing data. Prior to this study, 43 percent of IDX/VOW vendors surveyed were not aware of how pervasive bots and scraping were across the entire industry. 62 percent rated compliance with MLS rules as the most important factor in having IDX/VOW vendors implement an anti-scraping solution. Only 17 percent of IDX/VOW vendors are using a commercial service to protect against scraping. Some are using services with limited protection, especially against sophisticated web scraping bots. "Most IDX/VOW vendors are using reactive monitoring mechanisms and other outdated methods to stop scrapers, which are not effective against today's nefarious actors. For those attempting to implement proactive tools to combat scraping, almost all indicated they were using old IP-address-based methods, like rate limiting, CAPTCHAs, web application firewalls and IP blocking services. On their own, these methods are not effective," said Matt Cohen for Clareity Consulting. "Based on the near unanimous support for rule changes and a standard for testing compliance, combined with advances in affordable anti-scraper services available, there is a path forward for the industry to solve the scraping problem once and for all." Resources To download a copy of the full report, visit: 2015 Study: State of Web Scraping Data Theft Across Real Estate Websites & MLS Data To view a recent webinar, titled "Ensuring Real Estate Website Listing Data Security," visit: http://bit.ly/1Lu4ucQ For more information and case studies: http://resources.distilnetworks.com/h/c/175726-real-estate About Distil Networks Distil Networks, the global leader in bot detection and mitigation, is the first easy and accurate way to identify and police malicious website traffic, blocking 99.9% of bad bots without impacting legitimate users. Distil protects against web scraping, brute force attacks, competitive data mining, account hijacking, unauthorized vulnerability scans, spam, man-in-the-middle attacks and click fraud. Slash the high tax that bots place on your internal teams and web infrastructure by outsourcing the problem to the team with a maniacal focus on blocking malicious bots. For more information on Distil Networks, visit us at http://www.distilnetworks.com or follow @DISTIL on twitter. About Clareity Consulting Clareity Consulting provides management and information technology consulting for the real estate and related industries. Clients include software and service providers, Multiple Listing Services (MLS), real estate associations, franchises, and brokers. Specialties include strategic and business planning, system selection, compliance and risk audits, software product planning and review, and public speaking. More information: http://clareity.com