The Realities of Working with MLS Data

Guest contributor Chris Freeman of WolfNet Technologies says:

A couple years ago, I spoke at the Inman Connect real estate conference, in a session titled, “MLS Hell: Coping with Data Normalization in an Abnormal Industry.” From the title of the session alone, one can get a sense of the direction in which it went. WolfNet, being an IDX provider to the real estate industry, currently works with approximately 350 different MLSs, importing MLS data directly from each of the respective MLSs and making it searchable on the web for our Realtor clients. Other than a handful of MLSs which have merged, and two MLSs which share the same data format, all of the property data imports are completely different.

RETS, the Real Estate Transaction Standard (www.rets.org), as the name implies is a set of standards to which all MLSs make their information available to the real estate industry. Although the advent of RETS has been an improvement over the former method of FTPing flat data files, it has not resolved the primary issues of different data, data types and definitions between MLSs.

Ideally:

• RETS would make vendors’ data import scripts all the same
• All MLSs would use the same version of RETS
• All MLSs would use the same data column names and data types
• All MLS rules and regulations would be the same
• All MLSs would provide documentation for their data and data access
• All MLSs would notify vendors of changes before they occur
• RETS organization would provide very strong guidelines and recommendations
• We would have peace on Earth

In the real world, we have none of these things. The reality is that even though there is a data transport mechanism standard, RETS, the actual data format is not yet standardized between MLSs. Even such simple things such as number of bathrooms, property types, property styles and a listing’s street address are done in a multitude of ways and as such are not standardized. The applications used to connect to a RETS server also do not work across different RETS implementations. In the past, WolfNet was using five separate RETS connectors until we built our own application, which to date is the only known program that will connect to all RETS servers.

To be fair, this is not the fault of the RETS organization nor the MLSs; it’s simply that we are not at the final destination yet. Anyone who believes this ought to be simpler just needs to remember that there are more than 800 MLSs in the United States, not to mention the many MLS and IDX vendors. If you’ve ever been in a company meeting and tried to get a handful of people in one company to all agree, then try multiplying that by 800! There are some fantastic people working on RETS to make it more standardized in both the most recent versions as well as future versions. RETS has many advantages over the old methods, such as the ability to query only what has been updated or only what you need, one can query listing photos on a per listing basis and one can do live, up-to-the-minute queries instead of simply receiving updates once per day.

Michael Wurzer, from FlexMLS, posted on his blog some of the updates that RETS and its board members are currently looking at implementing:

“So, the writing on the wall seems clear. The biggest MLSs in the country are dead serious about data standards.” And “…defining data standards is only the first step and implementing them in the 800 MLSs around the country is a much more daunting task, requiring massive conversions or data mapping efforts by both MLSs and vendors.”

“The most important movement was the recommendation to the Board to form a workgroup to extend the existing standard names approved for RETS 1.8 into a data dictionary, including data types and enumerations.”

Currently, all of MLSs that WolfNet works with are using RETS 1.7.2 or earlier. Future iterations of RETS show promise that the challenges companies such as WolfNet currently face will be drastically reduced. As Michael points out, it is not just the creation of new data standards which will be a challenge but the implementation of those standards by the over 800 MLSs. An individual MLS by itself has little direct incentive to upgrade to the newest version of RETS as it can be a major undertaking in both time and money. The onus is not on an MLS to make data exchanges simpler on its vendors. If a given vendor wants to work with an MLS, the onus is entirely on that vendor to appropriately adapt the MLS data to their systems.

With that being said, there certainly are advantages to everyone of tightening the standards:

• Lower cost of entry for IDX and MLS vendors
• More competition amongst IDX and MLS vendors
• It would spur innovation due to lower costs
• Better return on investment for brokers, agents, MLSs and vendors
• Fewer issues between MLSs and vendors

There will come a day, in the not too distant future, when data standards will be tightened. Then, once all MLSs update to using this standard, we will see these advantages come to light. My own, educated guess for this timeframe is at least two to three years. Until that day comes, the reality is that if you work with multiple MLSs, you will keep a team of database developers very busy. Alternatively, one could work with a data partner who already standardizes MLS data, like WolfNet or one of our competitors. WolfNet’s database team has been standardizing data for more than a decade and we have built many checks and balances into our data imports over the years based on extensive experience.

In the spirit of full disclosure, the challenges outlined above are a competitive advantage for WolfNet. Since we do have a lot of experience mapping real estate data into our own standardized database format, we now have the process down to a science. However, we can all agree that a company ought to rise to the top based solely on the merits of their products and not due to an ability to navigate difficult data problems.

To view the original article, visit the WolfNet blog.