Sem categoria

The results of this research were applied to create the Portuguese Web Archive that is publicly available since January 2010. It supports full-text search over 1 billion contents archived from 1996 to 2010. The developed software is available as an open source project.

• Technical report
• Source-code

The Portuguese Web Archive will be presented at Jornadas FCCN on 7th and 8th of February 2012, with the following activities (in Portuguese):

• Demonstração do Arquivo da Web Portuguesa
• Arquivo da Web Portuguesa: Porquê? Para quê? Para quem?
• Zapping de Serviços FCCN

• We reached 1 200 million contents archived since 1996, 79% of them are indexed
• We save 62% of disk space by using DeDuplicator on our daily crawls
• We estimated to have disk space until 2017 to archive data and memory until 2014 to make it searchable

• Our search is supported by a distributed system composed by 111 multi-thread processes
• We have 61 computers:
  •  1.8 TB of memory;
  • 122 processors (488 cores);
  • 348 hard discs (320 TB).
• Try our search ove the past!

• 93% of the full-text searches performed on the Portuguese Web Archive were responded in less than 5 seconds.
• 95% of the URL searches were answered in less than 5 seconds.
• 73% of the user clicks are on the first page of results.
• We wrote 72 000 lines of code to improve the original search system based on the Archive-access project.
• Try our search!

The presentation focuses on the following points:

• The Web
• Web Archiving and crawlers
• Web characteristics and its evolution within 5 years
• The importance of the study on Web trends in the design of tools that process its data

Find out more:

• See video (on Educast)
• Download presentation slides (PDF)
• Read full paper (PDF)
• See other publications by the Portuguese Web Archive