craiggers

If you’ve ever used craigslist before then you know, it’s just not very good. That’s not to say you can’t find what you need on there. The site is full of amazing deals and goods and services of all kinds, but navigating it involves opening new browser tab after browser tab, going back and forth and generally losing your way.

 

For those of you who are tired of the craigslist user experience from circa 1996, head on over to craiggers, the site that lets you interact with Craigslist the way you ought to.

As the craiggers’ tagline says, the site is simply “craigslist data, better than craigslist!” It allows users a number of simple functions you’ve likely unconsciously wished for for years but didn’t even realize you were desperately missing. For example, the site separates navigation into a number of columns, so you don’t need to open listings in new tabs or hit the back and forward buttons all the time. Click on a result and it loads in the same page. Hit the down arrow or click on a different entry and it loads in the right most column without ever leaving the page.

Beyond navigation – which is quite an improvement already – craiggers adds on a new layer of functionality when it comes to searching. No longer do you have to search simply within a single geographic area. As the site points out, “there are cases when searching outside your immediate community benefits both seekers and providers,” giving the example of searching for a stolen bike or adopting a dog. When you search on craiggers, you can specify that you want to see results from neighboring locations and it will show you those as well.

Furthermore, if you wanted to search craigslist repeatedly, say for a job or an apartment, craiggers will not only let you save the search to repeat later, but it will also send you an email notification twice a day of results.

craiggers: An Example for Developers

For those of you out there interested in more than simply craiglist searches, there’s another interesting aspect to craiggers – it was built using the 3taps API. We firstwrote about 3taps last month when the company launched at the Data 2.0 conference, explaining how the company wanted to “democratize the exchange of data.”

Through the 3taps API, data from craigslist, eBay, Indeed, Etsy, Amazon and a host of other services is available in real-time, making mash-ups like this possible. Craiggers was built by the 3taps team as an example of the potential of its offering and we think it makes quite an argument.

Advertisements

Craigslist rentals and boundary layers API on rentrent.org (Alpha)

This API originated from my ‘Craigslist Rentals on Map’ websitewww.rentrent.org

As you can see, I haven’t put a lot of efforts to make this site pretty. I just wanted to make it usable. Making this API public is an effort to encourage others to create better websites than mine.

This API takes away the pain of crowling, mining, geocoding and indexing Craigslist data and provides very simple web service calls to fetch the data. This way you can focus on creating a great rentals classifieds application without worrying about GIS bit of it.
You can use this API with Google Maps, Microsoft Bing maps, Yahoo maps etc.


The API supports 2 calls:

Ads.aspx

Service URL: http://www.rentrent.org/RENT/Ads.aspx

Parameter Description
xmin Longitude (min)
ymin Latitude (min)
xmax Longitude (max)
ymax Latitude (max)
bd Number of Bedrooms
ba Number of Bathrooms
type 1: For room rentals
2: For apartment and houses
maxrecords If not passed, maxrecords is set to 250.
If you pass maxRecords=1500,
you can retrieve bulk data using one request.
throwErrorIfOverLimit If not passed, this is ‘true’
You can set throwErrorIfOverLimit=false to get the top ‘maxrecords’ instead of error.
callback Name of a javascript function you want to be called back.

Example URL:
http://www.rentrent.org/RENT/Ads.aspx?xmin=-118.01925659179687&ymin=33.71948521132481&xmax=-117.68142700195314&ymax=33.85644642218431&bd=&ba=&pets=-1&type=2&throwErrorIfOverLimit=false&callback=xxx

The output will be in JSON format. (If you need specific API, send an email on rentrentorg@gmail.com and I will try to speed up the documentation process.)


Map.aspx

Service URL:http://www.rentrent.org/BUY/Map.aspx

Parameter Description
TID Tile ID or Quad Key. (%4 in VE map)
GridX X value of a tile (For google map)
GridY y value of a tile (For google map)
GridZ Zoom level (For google map)
Layer Name of a layer

1. Neighborhoods
2. ElementarySchoolDistricts
3. SecondarySchoolDistricts
4. UnifiedSchoolDistricts

 

Example URL:

http://www.rentrent.org/BUY/Map.aspx?TID=0230121301213&Layers=Neighborhoods

hhttp://www.rentrent.org/BUY/Map.aspx?TID=0230121333&Layers=UnifiedSchoolDistricts

License/Disclaimer/Terms of Use:http://www.rentrent.org/BUY/Disclaimer.html

Craigslist Quietly Begins Testing Maps

Craigslist has just gotten a major cartographic upgrade. The popular classified advertising website has quietly begun testing embedded maps on ads for housing in the San Francisco Bay and Portland, Oregon areas.

Craigslist’s new housing ad maps use data from OpenStreetMap, a website that seeks to offer free and reliable location data compiled by volunteers around the world (similar in principle to Wikipedia), as the OpenStreetMap Foundation firstobserved on August 21.

“They appear to be hosting the maps themselves,” wrote Richard Fairhurst, a board member of the nonprofit OpenStreetMap Foundation, which coordinates the project, in an email to TPM.

The move comes just about a month after Craigslist filed a copyright infringement lawsuit against two other web startups, PadMapper and 3taps, for using Craigslist housing ad data on maps.

Specifically, PadMapper was sued for putting housing ads from Craigslist (as well as other brokers and realty listings websites) into a custom Google Maps view, showing the exact location of the houses listed in the ads as pins on a map. PadMapper said it obtained the data from 3taps, which in turn scrapes it from search engine listings of Craigslist posts.

In response, Craigslist reportedly began removing its post listings from search engines and introduced, then withdrew a new agreement with users to the exclusive rights to their ad content.

Craigslist was widely criticized by tech bloggers for its move to prosecute the two startups, all the more so because the company, which launched in 1995 and incorporated in 1999, has for most of its history been heralded as a beacon of Web openness and user-friendliness.

So there’s a certain irony to Craigslist suing other companies for creating maps with Craigslist ad data — claiming a breach of proprietary content — then turning around and using an open, crowd-sourced mapping solution to create its own maps.

It’s unclear whether Craigslist plans to expand the maps to other locations or post categories. Craigslist did not return request for comment in time for the publication of this post.

Still, Craigslist’s new maps test is significant for another reason: The company becomes the latest major tech brand to choose OpenStreetMap over Google Maps for reliable location data.AppleFoursquare and Wikipedia’s mobile app have all switched from Google Maps to OpenStreetMap this year alone. Major iPhone and iPad apps will also be changing from Google Maps to OpenStreetMap once it becomes the default on the iPhone and iPad this fall, with the release of Apple’s update mobile operating system, iOS6.

The defections are thought to be due to Google’s decision to begin charging heavy users of its Google Maps API (application programming interface — the code which allows third-party apps to use Google Maps data) in January. Google recently reduced the charges, with a company release saying Google had been “listening carefully to feedback.”

QQ图片20130916114856

 

https://www.padmapper.com/

3Taps: ‘We can get Craigslist data anyway

the aggregation site that has been battling Craigslist for the right to capture data from Craigslist and incorporate it into 3Taps’ larger database, today said it would continue compiling information from Craigslist indirectly.

A federal court judge ruled on Friday that 3Taps could be prohibited from accessing Craigslist through its own servers, under the federal Computer Fraud and Abuse Act.

“3Taps can continue to function because directly accessing (Craigslist’s) servers is only one of three ways in which the information in question can be obtained. The other two, crowdsourcing and public search results, require no such access to Craigslist’s servers,” the company said in a statement at 3Taps.com. 3Taps said it “respect[s] the court’s ruling … and will immediately cease all access to Craigslist’s servers.”

The company said Congress should “clarify the scope of the CFAA so that companies like Craigslist cannot use it as a tool to stifle competition, innovation and access to public websites.”

3Taps said Craigslist was using “an anticompetitive scheme to maintain its dominant positions in various online classified ad segments. Among other tactics, Craigslist threatens small, upstart competitors with baseless and exhaustive litigation to drive them out of business before the underlying substantive legal issues even can be addressed.”

(We’ve asked Craigslist for comment; no reply yet. We’ll update this post if we hear from them.)

The Electronic Frontier Foundation yesterday described the ruling as “narrow” and said it affirmed the right of individuals to view a publicly accessible website unless they have been served with notice to stop visiting the site, but said the use of IP blocking and cease-and-desist notices to keep people from viewing otherwise openly accessible websites sites could lead to “the potential for mischief … especially if [it] is done for arbitrary reasons.”

http://aimgroup.com/2013/08/20/3taps-we-can-get-craigslist-data-anyway/

How to scrape Craigslist data using WebHarvy ?

How to scrape Craigslist data using WebHarvy ?

WebHarvy can be used for scraping data from Craigslist pages. Data from various Craigslist categories can be easily extracted (scraped) using WebHarvy.

Scraped data can be easily exported as a spreadsheet file or to a database. There is no limit to the amount of data which can be extracted and exported. Listings which span across multiple pages can be easily extracted. WebHarvy also allows you to scrape listings from multiple geo locations within Craigslist.

Demo below shows how WebHarvy can be used to scrape data from Craigslist website:

https://www.webharvy.com/articles/scraping-craigslist.html