Difference between revisions of "TwitPic"

From Archiveteam
Jump to navigation Jump to search
(→‎How can I help?: invitation to IRC)
m (Reverted edits by Megalanya1 (talk) to last revision by Start)
(45 intermediate revisions by 8 users not shown)
Line 5: Line 5:
| description = TwitPic mainpage in 2011-01-12
| description = TwitPic mainpage in 2011-01-12
| URL = http://twitpic.com
| URL = http://twitpic.com
| project_status = {{closing}}
| project_status = {{specialcase}} (content to be possibly moved)
| archiving_status = {{in progress}}
| archiving_status = {{selfsaved}}
| irc = quitpic
| irc = quitpic
| tracker = [http://tracker.archiveteam.org/twitpicdisco twitpicdisco], [http://tracker.archiveteam.org/twitpic twitpic]
| tracker = [http://tracker.archiveteam.org/twitpicdisco twitpicdisco], [http://tracker.archiveteam.org/twitpic twitpic], [http://tracker.archiveteam.org/twitpic-cloudfront twitpic-cloudfront] '''[http://tracker.archiveteam.org/twitpic2 twitpic2]'''
| source = [https://github.com/ArchiveTeam/twitpic-discovery twitpic-discovery], [https://github.com/ArchiveTeam/twitpic-grab twitpic-grab], [https://github.com/ArchiveTeam/twitpic-items twitpic-items]
| source = [https://github.com/ArchiveTeam/twitpic-discovery twitpic-discovery], [https://github.com/ArchiveTeam/twitpic-grab twitpic-grab], [https://github.com/ArchiveTeam/twitpic-items twitpic-items], [https://github.com/ArchiveTeam/twitpic-cloudfront-grab twitpic-cloudfront-grab] '''[https://github.com/ArchiveTeam/twitpic-grab2 twitpic-grab2]'''
}}
}}
[[Image:Quitpic.png|center|100px]]


'''TwitPic''' is an image hosting service. The service is designed mainly for Twitter users - the images uploaded on the service are given short URLs for usage in Twitter posts. Twitter carries a 140-character post limit, the average Twitpic URL is 25/26 characters long.
'''TwitPic''' is an image hosting service. The service is designed mainly for Twitter users - the images uploaded on the service are given short URLs for usage in Twitter posts. Twitter carries a 140-character post limit, the average Twitpic URL is 25/26 characters long.


On September 4, 2014 TwitPic [http://blog.twitpic.com/2014/09/twitpic-is-shutting-down/ announced] they were shutting down on September 25.
On September 4, 2014 TwitPic [http://blog.twitpic.com/2014/09/twitpic-is-shutting-down/ announced] they were shutting down on September 25. On September 18, 2014, TwitPic [https://twitter.com/TwitPic/status/512705809696837632 announced] that they'd been acquired and would "live on". However, on October 16, 2014, Twitpic announced that "agreeable terms could not be met" and that the service would be [http://techcrunch.com/2014/10/16/twitpic-couldnt-find-an-acquirer-will-shut-down-after-all-on-oct-25th/ shutting down on October 25th]. On the shutdown date, it was made public that TwitPic [http://blog.twitpic.com/2014/10/twitpics-future/ would be taken by Twitter], thus the data would be kept, in read-only mode.


== Shutdown ==
== Shutdown ==
[[File:Twitpic-2014-10-16-at-8-25-08.png|thumb|right|400px|Twitpic's erratic demise]]


''Posted on September 4, 2014 by Noah Everett on [http://blog.twitpic.com blog.twitpic.com]:''
''Posted on September 4, 2014 by Noah Everett on [http://blog.twitpic.com blog.twitpic.com]:''


Twitpic will be shutting down September 25th. You will be able to export all your photos and videos. We’ll let everyone know when this feature is live in the next few days.
"Twitpic will be shutting down September 25th. You will be able to export all your photos and videos. We’ll let everyone know when this feature is live in the next few days.


This is an unexpected and hard announcement for us to make and we want to lay out what led us to this decision.
This is an unexpected and hard announcement for us to make and we want to lay out what led us to this decision.
Line 32: Line 36:
Unfortunately we do not have the resources to fend off a large company like Twitter to maintain our mark which we believe whole heartedly is rightfully ours. Therefore, we have decided to shut down Twitpic.
Unfortunately we do not have the resources to fend off a large company like Twitter to maintain our mark which we believe whole heartedly is rightfully ours. Therefore, we have decided to shut down Twitpic.


On a personal note I (@noaheverett) want to thank you for letting us be a part of your life and helping you share your experiences over the past 6 years, it’s truly been an honor. I have learned so much through running Twitpic over the years. Through the many mistakes I’ve made and lessons learned, to the bad days and the great days. Thank you again everyone…I will miss and cherish the days of Twitpic we had together.
On a personal note I (@noaheverett) want to thank you for letting us be a part of your life and helping you share your experiences over the past 6 years, it’s truly been an honor. I have learned so much through running Twitpic over the years. Through the many mistakes I’ve made and lessons learned, to the bad days and the great days. Thank you again everyone…I will miss and cherish the days of Twitpic we had together."
 
=== Won't (?) shut down ===
 
Twitpic [https://twitter.com/TwitPic/status/512705809696837632 writes] on Twitter, on September 18, 2014:
 
:"We're happy to announce we've been acquired and Twitpic will live on! We will post more details as we can disclose them"
 
However, ArchiveTeam goes on downloading TwitPic, for safety.
 
=== IT INDEED WILL ===
 
'''UPDATE''' on [http://blog.twitpic.com blog.twitpic.com]:
 
:"It’s with a heavy heart that I announce again that Twitpic will be shutting down on October 25th. We worked through a handful of potential acquirers and exhausted all potential options. We were almost certain we had found a new home for Twitpic (hence our previous tweet), but agreeable terms could not be met. Normally we wouldn’t announce something like that prematurely but we were hoping to let our users know as soon as possible that Twitpic was living on.
 
:I’m sincerely sorry (and embarrassed) for the circumstances leading up to this, from our initial shutdown announcement to an acquisition false alarm.
 
:You can export your data and photos at: http://twitpic.com/account/settings "
 
=== But wait! There's more! ===
 
On October 17th, 2014, Twitpic began blocking public access to images,<ref>https://twitter.com/textfiles/status/523160196672409600</ref> replacing them with a shutdown notice. Comments are still available for now, but the images are not.
 
=== Saved by Twitter; full access again ===
 
''Noah Everett on October 25, 2014, on blog.twitpic.com:''
 
:"[...] I’m happy to announce that we have reached an agreement with Twitter to give them the Twitpic domain and photo archive, thus keeping the photos and links alive for the time being. [...]"
 
Read the full entry: http://blog.twitpic.com/2014/10/twitpics-future/
 
At the same time, public access to the pictures was given back. So the site basically became what is was like. However, we don't know anything about the potential changes that may occur upon the agreement. '''ArchiveTeam decided to make a full copy of TwitPic as it looks like now.'''


== Site structure ==
== Site structure ==
Line 38: Line 74:
Image page urls:
Image page urls:


http://twitpic.com/******
* http://twitpic.com/******
http://twitpic.com/*****
* http://twitpic.com/*****
http://twitpic.com/****
* http://twitpic.com/****
http://twitpic.com/***
* http://twitpic.com/***
http://twitpic.com/**
* http://twitpic.com/**
http://twitpic.com/*
* http://twitpic.com/*
 
where * = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z
where * = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z


where ****** consists of up to 6 alphanumeric characters. Leading zeros are irrelevant, e.g.: /000joe = /0joe = /joe. Like incremental numbers in base-36 numeral system.
where ****** consists of up to 6 alphanumeric characters. Leading zeros are irrelevant, e.g.: /000joe = /0joe = /joe. Like incremental numbers in base-36 numeral system.


== Progress ==
* Original image URL: https://d3j5vwomefv46c.cloudfront.net/photos/large/1165.jpg
* Video: https://d3j5vwomefv46c.cloudfront.net/photos/video/238148666.mp4
* Post-shutdown URL: https://dn3pm25xmtlyu.cloudfront.net/photos/large/2.jpg?1202229769&Expires=1414338482&Signature=VngmKpZJc~f~JKTwC6Xi-8~TUDVVaElfqQ6Xn3mvBhS9gDEOlmx8xVCj~Vve35fx1n5jXrIhHPIm-lYBDkSjVjkIo-IZuxGRGplMOcqmexXjVROM7OPKbdhZ-2XaWgcgaieJDNHaB1LyxwGsbIVyg~JuadStL-jGw6Kby-lL5bY_&Key-Pair-Id=APKAIYVGSUJFNRFZBBTA


Right now [http://tracker.archiveteam.org/twitpicdisco content discovery] is in progress, which will be followed by the [http://tracker.archiveteam.org/twitpic actual grabbing process].
== Archiving report ==


== How can I help? ==
=== Phase 1: content discovery ===


You can either start up a [[Warrior]] (and select ''TwitPic Phase 1'', or rather ''ArchiveTeam's Choice''), or run the scripts manually (see ''Project source'' above in the box).
From September 5 to 6, until ArchiveTeam got banned, ~41 million of the possible ~900 million urls were discovered. The discovery was suspended.


Please note that one discovery item contains 1.000 urls and so takes some time to finish (although items are very little in size). Additionally, twitpic.com seems to have relatively low throughput capacity, thus rate limiting is active to prevent overloading.
On September 6th, someone claiming to be Noah Everett showed up in #quitpic<ref>http://paste.archivingyoursh.it/raw/xesequhogi</ref>:


You should also join the dedicated IRC channel '''#quitpic''', to catch the latest news about the project and its status, and there you can also put your questions left unanswered after studying this wiki (or the script instructions). You can use the web interface at http://chat.efnet.org:9090, or, if you prefer your favourite IRC client, then your destination is irc://irc.efnet.org/quitpic.
<pre>
[16:21:14] <n00b957> hey guys
[16:21:16] <n00b957> Noah Everett here
[16:21:26] <n00b957> noticed the site was really bogging down due to ArchiveTeam requersts
[16:21:30] <n00b957> *requests
[16:21:55] <n00b957> didn't know what it was at first so we blocked it to continue normal site operations and users can get their data easily
[16:22:27] <n00b957> just wanted to give a heads up so you don't think we are trying to be malicious
[16:23:00] <n00b957> we're working on getting our export tool out the door right now
[16:23:14] <n00b957> I'd like to let our users get their data off the site via that first as quickly as possible
</pre>
 
Unfortunately, he left #quitpic shortly afterwards and has not returned any of Archive Team's repeated inquiries about archiving Twitpic.
 
=== Phase 2: content grab ===
 
After some testing, actual content grab began on September 14.
 
On October 25, 2014, when Noah Everett announced that TwitPic would be saved by Twitter, ArchiveTeam decided to start a new, full, fresh grab of the site as it looked like before any changes caused by the new situation.
 
=== A brand new grab ===
 
Afterwards, ArchiveTeam got the images and image pages. Other data (tags, albums etc.) was also planned to be taken, but, in the end, wasn't done.<sup>''[confirmation needed]''</sup>


== Archives ==
== Archives ==


Coming soon.
Archives are stored in [[WARC]] format available at [https://archive.org/details/archiveteam_twitpic archiveteam_twitpic].
 
== Download Your Data ==
 
'''"You can export your data and photos at: http://twitpic.com/account/settings"'''
 
When it's about ''your'' content, don't rely on ArchiveTeam's archives, as they may be incomplete, and are not made in a way that a single user's content can be extracted from them. '''Use the export-tool!'''
 
=== Export Tool Bugs ===
 
Twitpic's export tool is buggy, handing out seemingly empty zip files<ref>https://news.ycombinator.com/item?id=8473393</ref> and 503 errors.<ref>https://twitter.com/textfiles/status/522837349676236801</ref> The empty zip file problem can sometimes be fixed:
 
The problem is twofold.  If the problem is on a non-Windows computer, it is probably a corrupted download (which happens way too often).  On Windows, the built in zip file handler is not able to reliably handle zip files.  [http://www.7-zip.org/ 7-zip] seems to have the most success with the zip file but others have worked as well.
 
General process to follow:
 
<ol>
<li>Download and install 7z</li>
<li>Download the zip file and rename it something short.</li>
<li>Open a command prompt.</li>
<li>run the command 7z t zipfilename.zip</li>
<li>If it tests successfully run 7z x zipfilename.zip</li>
<li>Browse to the photo directory.</li>
<li>Pictures should be visible and a text file with the metadata.</li>
</ol>


== Downloaders ==
== Downloaders ==
* [http://code.google.com/p/emijrp/source/browse/trunk/scrapers/twitpic.py Downloader by tag] (it saves the full resolution image and metadata: uploader, date and description)
* [http://code.google.com/p/emijrp/source/browse/trunk/scrapers/twitpic.py Downloader by tag] (it saves the full resolution image and metadata: uploader, date and description)
== References ==
<references/>


== External links ==
== External links ==
* http://twitpic.com
* http://twitpic.com
* [http://mashable.com/2008/02/05/twit-pic/ "Twit Pic. It's For Twitter Pictures, Of Course."]
* [http://mixergy.com/interviews/twitpic-noah-everett/ "How A Spare Computer Became Twitpic, A $1.5+ Million A Year Twitter Success Story – with Noah Everett"]
* [https://twitter.com/TwitPic/status/512705809696837632 "We're happy to announce we've been acquired and Twitpic will live on! ..."]
* [http://techcrunch.com/2014/10/16/twitpic-couldnt-find-an-acquirer-will-shut-down-after-all-on-oct-25th/ "Twitpic Couldn’t Find An Acquirer, Will Shut Down After All On Oct 25th"]
* [https://www.youtube.com/watch?v=S8JjLJKMmNE#t=4517 "Documentar y Archivar el Internet. Mesa 2 SIPAD"]: Comments on TwitPic at 1:15:17
=== Archive Team In The Press ===
* [http://globalnews.ca/news/1633807/800-million-twitpic-photos-to-vanish-from-the-web-saturday/ "800 million Twitpic photos to vanish from the web Saturday" (w/ Jason Scott excerpts)]
* [http://www.geekwire.com/2014/archive-team-twitpic-blocking-us-downloading-photos-shutdown/ "Lost forever? Archive Team says TwitPic is blocking photo downloads before shutdown"]
* [http://associationsnow.com/2014/10/twitpic-online-startup-fails/ "Twitpic: What We Lose When an Online Startup Fails"]
* [http://www.theglobeandmail.com/technology/digital-culture/the-race-to-archive-twitpic-before-800-million-pictures-vanish/article21199755/ "The race to archive Twitpic before 800 million pictures vanish"]


{{Navigation box}}
{{Navigation box}}


[[Category:Image hosting services]]
[[Category:Image hosting]]

Revision as of 16:07, 17 January 2017

TwitPic
TwitPic logo
TwitPic mainpage in 2011-01-12
TwitPic mainpage in 2011-01-12
URL http://twitpic.com
Status Special case (content to be possibly moved)
Archiving status Saved by itself
Archiving type Unknown
Project source twitpic-discovery, twitpic-grab, twitpic-items, twitpic-cloudfront-grab twitpic-grab2
Project tracker twitpicdisco, twitpic, twitpic-cloudfront twitpic2
IRC channel #quitpic (on hackint)
Quitpic.png

TwitPic is an image hosting service. The service is designed mainly for Twitter users - the images uploaded on the service are given short URLs for usage in Twitter posts. Twitter carries a 140-character post limit, the average Twitpic URL is 25/26 characters long.

On September 4, 2014 TwitPic announced they were shutting down on September 25. On September 18, 2014, TwitPic announced that they'd been acquired and would "live on". However, on October 16, 2014, Twitpic announced that "agreeable terms could not be met" and that the service would be shutting down on October 25th. On the shutdown date, it was made public that TwitPic would be taken by Twitter, thus the data would be kept, in read-only mode.

Shutdown

Twitpic's erratic demise

Posted on September 4, 2014 by Noah Everett on blog.twitpic.com:

"Twitpic will be shutting down September 25th. You will be able to export all your photos and videos. We’ll let everyone know when this feature is live in the next few days.

This is an unexpected and hard announcement for us to make and we want to lay out what led us to this decision.

A few weeks ago Twitter contacted our legal demanding that we abandon our trademark application or risk losing access to their API. This came as a shock to us since Twitpic has been around since early 2008, and our trademark application has been in the USPTO since 2009.

Here is some backstory on the history of our trademark:

We originally filed for our trademark in 2009 and our first use in commerce dates back to February 2008 when we launched. We encountered several hurdles and difficulties in getting our trademark approved even though our first use in commerce predated other applications, but we worked through each challenge and in fact had just recently finished the last one. During the “published for opposition” phase of the trademark is when Twitter reached out to our counsel and implied we could be denied access to their API if we did not give up our mark.

Unfortunately we do not have the resources to fend off a large company like Twitter to maintain our mark which we believe whole heartedly is rightfully ours. Therefore, we have decided to shut down Twitpic.

On a personal note I (@noaheverett) want to thank you for letting us be a part of your life and helping you share your experiences over the past 6 years, it’s truly been an honor. I have learned so much through running Twitpic over the years. Through the many mistakes I’ve made and lessons learned, to the bad days and the great days. Thank you again everyone…I will miss and cherish the days of Twitpic we had together."

Won't (?) shut down

Twitpic writes on Twitter, on September 18, 2014:

"We're happy to announce we've been acquired and Twitpic will live on! We will post more details as we can disclose them"

However, ArchiveTeam goes on downloading TwitPic, for safety.

IT INDEED WILL

UPDATE on blog.twitpic.com:

"It’s with a heavy heart that I announce again that Twitpic will be shutting down on October 25th. We worked through a handful of potential acquirers and exhausted all potential options. We were almost certain we had found a new home for Twitpic (hence our previous tweet), but agreeable terms could not be met. Normally we wouldn’t announce something like that prematurely but we were hoping to let our users know as soon as possible that Twitpic was living on.
I’m sincerely sorry (and embarrassed) for the circumstances leading up to this, from our initial shutdown announcement to an acquisition false alarm.
You can export your data and photos at: http://twitpic.com/account/settings "

But wait! There's more!

On October 17th, 2014, Twitpic began blocking public access to images,[1] replacing them with a shutdown notice. Comments are still available for now, but the images are not.

Saved by Twitter; full access again

Noah Everett on October 25, 2014, on blog.twitpic.com:

"[...] I’m happy to announce that we have reached an agreement with Twitter to give them the Twitpic domain and photo archive, thus keeping the photos and links alive for the time being. [...]"

Read the full entry: http://blog.twitpic.com/2014/10/twitpics-future/

At the same time, public access to the pictures was given back. So the site basically became what is was like. However, we don't know anything about the potential changes that may occur upon the agreement. ArchiveTeam decided to make a full copy of TwitPic as it looks like now.

Site structure

Image page urls:

where * = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z

where ****** consists of up to 6 alphanumeric characters. Leading zeros are irrelevant, e.g.: /000joe = /0joe = /joe. Like incremental numbers in base-36 numeral system.

Archiving report

Phase 1: content discovery

From September 5 to 6, until ArchiveTeam got banned, ~41 million of the possible ~900 million urls were discovered. The discovery was suspended.

On September 6th, someone claiming to be Noah Everett showed up in #quitpic[2]:

[16:21:14] <n00b957> hey guys
[16:21:16] <n00b957> Noah Everett here
[16:21:26] <n00b957> noticed the site was really bogging down due to ArchiveTeam requersts
[16:21:30] <n00b957> *requests
[16:21:55] <n00b957> didn't know what it was at first so we blocked it to continue normal site operations and users can get their data easily
[16:22:27] <n00b957> just wanted to give a heads up so you don't think we are trying to be malicious
[16:23:00] <n00b957> we're working on getting our export tool out the door right now
[16:23:14] <n00b957> I'd like to let our users get their data off the site via that first as quickly as possible

Unfortunately, he left #quitpic shortly afterwards and has not returned any of Archive Team's repeated inquiries about archiving Twitpic.

Phase 2: content grab

After some testing, actual content grab began on September 14.

On October 25, 2014, when Noah Everett announced that TwitPic would be saved by Twitter, ArchiveTeam decided to start a new, full, fresh grab of the site as it looked like before any changes caused by the new situation.

A brand new grab

Afterwards, ArchiveTeam got the images and image pages. Other data (tags, albums etc.) was also planned to be taken, but, in the end, wasn't done.[confirmation needed]

Archives

Archives are stored in WARC format available at archiveteam_twitpic.

Download Your Data

"You can export your data and photos at: http://twitpic.com/account/settings"

When it's about your content, don't rely on ArchiveTeam's archives, as they may be incomplete, and are not made in a way that a single user's content can be extracted from them. Use the export-tool!

Export Tool Bugs

Twitpic's export tool is buggy, handing out seemingly empty zip files[3] and 503 errors.[4] The empty zip file problem can sometimes be fixed:

The problem is twofold. If the problem is on a non-Windows computer, it is probably a corrupted download (which happens way too often). On Windows, the built in zip file handler is not able to reliably handle zip files. 7-zip seems to have the most success with the zip file but others have worked as well.

General process to follow:

  1. Download and install 7z
  2. Download the zip file and rename it something short.
  3. Open a command prompt.
  4. run the command 7z t zipfilename.zip
  5. If it tests successfully run 7z x zipfilename.zip
  6. Browse to the photo directory.
  7. Pictures should be visible and a text file with the metadata.

Downloaders

  • Downloader by tag (it saves the full resolution image and metadata: uploader, date and description)

References

External links

Archive Team In The Press