Dynamic Boundaries

Hypothesis Statement:

The boundaries of a city are not fixed, they are dynamic. The hard edge boundary lines illustrated in traditional municipal maps do not reflect the lively nature of a city. By querying Foursquare “here/now” data and employing clustering algorithms, our prototype groups Foursquare venues together based on location and activity levels. By identifying and mapping these clusters through time, this tool helps to reveal the dynamic patterns and processes of boundary formation, mapping out opportunities for effective intervention.


Process Diagram


The Livehoods Project: Utilizing Social Media to Understand the Dynamics of a City

Studying the social dynamics of a city on a large scale has tra- ditionally been a challenging endeavor, requiring long hours of observation and interviews, usually resulting in only a par- tial depiction of reality. At the same time, the boundaries of municipal organizational units, such as neighborhoods and districts, are largely statically defined by the city government and do not always reflect the character of life in these ar- eas. To address both difficulties, we introduce a clustering model and research methodology for studying the structure and composition of a city based on the social media its res- idents generate. We use data from approximately 18 million check-ins collected from users of a location-based online so- cial network. The resulting clusters, which we call Livehoods, are representations of the dynamic urban areas that comprise the city. We take an interdisciplinary approach to validating these clusters, interviewing 27 residents of Pittsburgh, PA, to see how their perceptions of the city project onto our findings there. Our results provide strong support for the discovered clusters, showing how Livehoods reveal the distinctly charac- terized areas of the city and the forces that shape them.


Neighborhood boundaries based on social media activity

Researchers at the School of Computer Science at Carnegie Mellon University investigate the structure of cities in Livehoods, using foursquare check-ins.

The hypothesis underlying our work is that the character of an urban area is defined not just by the the types of places found there, but also by the people who make the area part of their daily routine. To explore this hypothesis, given data from over 18 million foursquare check-ins, we introduce a model that groups nearby venues into areas based on patterns in the set of people who check-in to them. By examining patterns in these check-ins, we can learn about the different areas that comprise the city, allowing us to study the social dynamics, structure, and character of cities on a large scale.

It’s most interesting when you click on location dots. A Livehood is highlighted and a panel on the top right tells you what the neighborhood is like, related neighborhoods, and provides stats like hourly and daily pulse and a breakdown location categories (for example, food and nightlife). Does foursquare have anything like this tied into their system? They should if they don’t.

There’s only maps for San Francisco, New York City, and Pittsburgh right now, but I’m sure there are more to come.

Want more on the clustering behind the maps? Here’s the paper [pdf].livehoods_icwsm12livehoods


If you’ve ever used craigslist before then you know, it’s just not very good. That’s not to say you can’t find what you need on there. The site is full of amazing deals and goods and services of all kinds, but navigating it involves opening new browser tab after browser tab, going back and forth and generally losing your way.


For those of you who are tired of the craigslist user experience from circa 1996, head on over to craiggers, the site that lets you interact with Craigslist the way you ought to.

As the craiggers’ tagline says, the site is simply “craigslist data, better than craigslist!” It allows users a number of simple functions you’ve likely unconsciously wished for for years but didn’t even realize you were desperately missing. For example, the site separates navigation into a number of columns, so you don’t need to open listings in new tabs or hit the back and forward buttons all the time. Click on a result and it loads in the same page. Hit the down arrow or click on a different entry and it loads in the right most column without ever leaving the page.

Beyond navigation – which is quite an improvement already – craiggers adds on a new layer of functionality when it comes to searching. No longer do you have to search simply within a single geographic area. As the site points out, “there are cases when searching outside your immediate community benefits both seekers and providers,” giving the example of searching for a stolen bike or adopting a dog. When you search on craiggers, you can specify that you want to see results from neighboring locations and it will show you those as well.

Furthermore, if you wanted to search craigslist repeatedly, say for a job or an apartment, craiggers will not only let you save the search to repeat later, but it will also send you an email notification twice a day of results.

craiggers: An Example for Developers

For those of you out there interested in more than simply craiglist searches, there’s another interesting aspect to craiggers – it was built using the 3taps API. We firstwrote about 3taps last month when the company launched at the Data 2.0 conference, explaining how the company wanted to “democratize the exchange of data.”

Through the 3taps API, data from craigslist, eBay, Indeed, Etsy, Amazon and a host of other services is available in real-time, making mash-ups like this possible. Craiggers was built by the 3taps team as an example of the potential of its offering and we think it makes quite an argument.

Craigslist rentals and boundary layers API on rentrent.org (Alpha)

This API originated from my ‘Craigslist Rentals on Map’ websitewww.rentrent.org

As you can see, I haven’t put a lot of efforts to make this site pretty. I just wanted to make it usable. Making this API public is an effort to encourage others to create better websites than mine.

This API takes away the pain of crowling, mining, geocoding and indexing Craigslist data and provides very simple web service calls to fetch the data. This way you can focus on creating a great rentals classifieds application without worrying about GIS bit of it.
You can use this API with Google Maps, Microsoft Bing maps, Yahoo maps etc.

The API supports 2 calls:


Service URL: http://www.rentrent.org/RENT/Ads.aspx

Parameter Description
xmin Longitude (min)
ymin Latitude (min)
xmax Longitude (max)
ymax Latitude (max)
bd Number of Bedrooms
ba Number of Bathrooms
type 1: For room rentals
2: For apartment and houses
maxrecords If not passed, maxrecords is set to 250.
If you pass maxRecords=1500,
you can retrieve bulk data using one request.
throwErrorIfOverLimit If not passed, this is ‘true’
You can set throwErrorIfOverLimit=false to get the top ‘maxrecords’ instead of error.
callback Name of a javascript function you want to be called back.

Example URL:

The output will be in JSON format. (If you need specific API, send an email on rentrentorg@gmail.com and I will try to speed up the documentation process.)


Service URL:http://www.rentrent.org/BUY/Map.aspx

Parameter Description
TID Tile ID or Quad Key. (%4 in VE map)
GridX X value of a tile (For google map)
GridY y value of a tile (For google map)
GridZ Zoom level (For google map)
Layer Name of a layer

1. Neighborhoods
2. ElementarySchoolDistricts
3. SecondarySchoolDistricts
4. UnifiedSchoolDistricts


Example URL:



License/Disclaimer/Terms of Use:http://www.rentrent.org/BUY/Disclaimer.html

Craigslist Quietly Begins Testing Maps

Craigslist has just gotten a major cartographic upgrade. The popular classified advertising website has quietly begun testing embedded maps on ads for housing in the San Francisco Bay and Portland, Oregon areas.

Craigslist’s new housing ad maps use data from OpenStreetMap, a website that seeks to offer free and reliable location data compiled by volunteers around the world (similar in principle to Wikipedia), as the OpenStreetMap Foundation firstobserved on August 21.

“They appear to be hosting the maps themselves,” wrote Richard Fairhurst, a board member of the nonprofit OpenStreetMap Foundation, which coordinates the project, in an email to TPM.

The move comes just about a month after Craigslist filed a copyright infringement lawsuit against two other web startups, PadMapper and 3taps, for using Craigslist housing ad data on maps.

Specifically, PadMapper was sued for putting housing ads from Craigslist (as well as other brokers and realty listings websites) into a custom Google Maps view, showing the exact location of the houses listed in the ads as pins on a map. PadMapper said it obtained the data from 3taps, which in turn scrapes it from search engine listings of Craigslist posts.

In response, Craigslist reportedly began removing its post listings from search engines and introduced, then withdrew a new agreement with users to the exclusive rights to their ad content.

Craigslist was widely criticized by tech bloggers for its move to prosecute the two startups, all the more so because the company, which launched in 1995 and incorporated in 1999, has for most of its history been heralded as a beacon of Web openness and user-friendliness.

So there’s a certain irony to Craigslist suing other companies for creating maps with Craigslist ad data — claiming a breach of proprietary content — then turning around and using an open, crowd-sourced mapping solution to create its own maps.

It’s unclear whether Craigslist plans to expand the maps to other locations or post categories. Craigslist did not return request for comment in time for the publication of this post.

Still, Craigslist’s new maps test is significant for another reason: The company becomes the latest major tech brand to choose OpenStreetMap over Google Maps for reliable location data.AppleFoursquare and Wikipedia’s mobile app have all switched from Google Maps to OpenStreetMap this year alone. Major iPhone and iPad apps will also be changing from Google Maps to OpenStreetMap once it becomes the default on the iPhone and iPad this fall, with the release of Apple’s update mobile operating system, iOS6.

The defections are thought to be due to Google’s decision to begin charging heavy users of its Google Maps API (application programming interface — the code which allows third-party apps to use Google Maps data) in January. Google recently reduced the charges, with a company release saying Google had been “listening carefully to feedback.”