Eyebrowse/ApacheLists

Goal: Setup indexing for apache.org lists

Status:

• DONE announce

• DONE mirrors

• On nagoya as qmlist user
  • Make new mbox archive area(s)

{{{DONE mkdir -p /opt/ezmlm/archive/apache.org/announce ["DONE"] mkdir -p /opt/ezmlm/archive/apache.org/mirrors }}}

• Copy over initial mbox file(s):

{{{ *** daedalus:/www/tlp.apache.org/mail/list has the mbox files in compressed format. They need to be uncompressed before running eyebrowse.

["DONE"] cd /opt/ezmlm/archive/apache.org/announce ["DONE"] wget apache.org/mail/announce/199910.gz ["DONE"] wget apache.org/mail/announce/199911.gz ["DONE"] wget apache.org/mail/announce/200002.gz ["DONE"] wget apache.org/mail/announce/200003.gz ["DONE"] wget apache.org/mail/announce/200004.gz ["DONE"] wget apache.org/mail/announce/200005.gz ["DONE"] wget apache.org/mail/announce/200006.gz ["DONE"] wget apache.org/mail/announce/200007.gz ["DONE"] wget apache.org/mail/announce/200008.gz ["DONE"] wget apache.org/mail/announce/200009.gz ["DONE"] wget apache.org/mail/announce/200010.gz ["DONE"] wget apache.org/mail/announce/200011.gz ["DONE"] wget apache.org/mail/announce/200012.gz ["DONE"] wget apache.org/mail/announce/200101.gz ["DONE"] wget apache.org/mail/announce/200103.gz ["DONE"] wget apache.org/mail/announce/200104.gz ["DONE"] wget apache.org/mail/announce/200105.gz ["DONE"] wget apache.org/mail/announce/200106.gz ["DONE"] wget apache.org/mail/announce/200107.gz ["DONE"] wget apache.org/mail/announce/200108.gz ["DONE"] wget apache.org/mail/announce/200109.gz ["DONE"] wget apache.org/mail/announce/200110.gz ["DONE"] wget apache.org/mail/announce/200111.gz ["DONE"] wget apache.org/mail/announce/200112.gz ["DONE"] wget apache.org/mail/announce/200201.gz ["DONE"] wget apache.org/mail/announce/200202.gz ["DONE"] wget apache.org/mail/announce/200203.gz ["DONE"] wget apache.org/mail/announce/200204.gz ["DONE"] wget apache.org/mail/announce/200205.gz ["DONE"] wget apache.org/mail/announce/200206.gz ["DONE"] wget apache.org/mail/announce/200207.gz ["DONE"] wget apache.org/mail/announce/200208.gz ["DONE"] wget apache.org/mail/announce/200209.gz ["DONE"] wget apache.org/mail/announce/200210.gz ["DONE"] wget apache.org/mail/announce/200211.gz ["DONE"] wget apache.org/mail/announce/200212.gz ["DONE"] wget apache.org/mail/announce/200301.gz ["DONE"] wget apache.org/mail/announce/200302.gz ["DONE"] wget apache.org/mail/announce/200303.gz ["DONE"] wget apache.org/mail/announce/200304.gz ["DONE"] wget apache.org/mail/announce/200305.gz ["DONE"] wget apache.org/mail/announce/200306.gz ["DONE"] wget apache.org/mail/announce/200307.gz ["DONE"] gunzip *.gz

["DONE"] cd /opt/ezmlm/archive/apache.org/mirrors ["DONE"] wget apache.org/mail/mirrors/199704.gz ["DONE"] wget apache.org/mail/mirrors/199706.gz ["DONE"] wget apache.org/mail/mirrors/199707.gz ["DONE"] wget apache.org/mail/mirrors/199708.gz ["DONE"] wget apache.org/mail/mirrors/199710.gz ["DONE"] wget apache.org/mail/mirrors/199711.gz ["DONE"] wget apache.org/mail/mirrors/199802.gz ["DONE"] wget apache.org/mail/mirrors/199804.gz ["DONE"] wget apache.org/mail/mirrors/199806.gz ["DONE"] wget apache.org/mail/mirrors/199807.gz ["DONE"] wget apache.org/mail/mirrors/199808.gz ["DONE"] wget apache.org/mail/mirrors/199809.gz ["DONE"] wget apache.org/mail/mirrors/199810.gz ["DONE"] wget apache.org/mail/mirrors/199811.gz ["DONE"] wget apache.org/mail/mirrors/199812.gz ["DONE"] wget apache.org/mail/mirrors/199903.gz ["DONE"] wget apache.org/mail/mirrors/199904.gz ["DONE"] wget apache.org/mail/mirrors/199908.gz ["DONE"] wget apache.org/mail/mirrors/200002.gz ["DONE"] wget apache.org/mail/mirrors/200003.gz ["DONE"] wget apache.org/mail/mirrors/200004.gz ["DONE"] wget apache.org/mail/mirrors/200005.gz ["DONE"] wget apache.org/mail/mirrors/200006.gz ["DONE"] wget apache.org/mail/mirrors/200007.gz ["DONE"] wget apache.org/mail/mirrors/200008.gz ["DONE"] wget apache.org/mail/mirrors/200009.gz ["DONE"] wget apache.org/mail/mirrors/200010.gz ["DONE"] wget apache.org/mail/mirrors/200011.gz ["DONE"] wget apache.org/mail/mirrors/200012.gz ["DONE"] wget apache.org/mail/mirrors/200101.gz ["DONE"] wget apache.org/mail/mirrors/200102.gz ["DONE"] wget apache.org/mail/mirrors/200103.gz ["DONE"] wget apache.org/mail/mirrors/200104.gz ["DONE"] wget apache.org/mail/mirrors/200105.gz ["DONE"] wget apache.org/mail/mirrors/200106.gz ["DONE"] wget apache.org/mail/mirrors/200107.gz ["DONE"] wget apache.org/mail/mirrors/200108.gz ["DONE"] wget apache.org/mail/mirrors/200109.gz ["DONE"] wget apache.org/mail/mirrors/200110.gz ["DONE"] wget apache.org/mail/mirrors/200111.gz ["DONE"] wget apache.org/mail/mirrors/200112.gz ["DONE"] wget apache.org/mail/mirrors/200201.gz ["DONE"] wget apache.org/mail/mirrors/200202.gz ["DONE"] wget apache.org/mail/mirrors/200203.gz ["DONE"] wget apache.org/mail/mirrors/200204.gz ["DONE"] wget apache.org/mail/mirrors/200205.gz ["DONE"] wget apache.org/mail/mirrors/200206.gz ["DONE"] wget apache.org/mail/mirrors/200207.gz ["DONE"] wget apache.org/mail/mirrors/200208.gz ["DONE"] wget apache.org/mail/mirrors/200209.gz ["DONE"] wget apache.org/mail/mirrors/200210.gz ["DONE"] wget apache.org/mail/mirrors/200211.gz ["DONE"] wget apache.org/mail/mirrors/200212.gz ["DONE"] wget apache.org/mail/mirrors/200301.gz ["DONE"] wget apache.org/mail/mirrors/200302.gz ["DONE"] wget apache.org/mail/mirrors/200303.gz ["DONE"] wget apache.org/mail/mirrors/200304.gz ["DONE"] wget apache.org/mail/mirrors/200305.gz ["DONE"] wget apache.org/mail/mirrors/200306.gz ["DONE"] wget apache.org/mail/mirrors/200307.gz ["DONE"] gunzip *.gz }}}

• On nagoya as regular user
  • Register new lists

{{{ export WEB_APP_DIR='/opt/tomcat/webapps/eyebrowse'

• export EYEBROWSE="${WEB_APP_DIR}/WEB-INF/bin/eyebrowse --props=${WEB_APP_DIR}/WEB-INF/classes/eyebrowse.properties"

["DONE"] ${EYEBROWSE} --command=addlist --listname='announce@apache.org' --descr='Apache Announcements' --indexdir="${WEB_APP_DIR}/index/apache.org/announce" ["DONE"] ${EYEBROWSE} --command=addlist --listname='mirrors@apache.org' --descr='Apache Mirroring' --indexdir="${WEB_APP_DIR}/index/apache.org/mirrors" }}}

• Add initial mbox file to the database, update the database and Lucene indices:
• ~noel/update-list implements addlistfile, updatelist, and updateindex in order
  • Repeat for each initial mbox for each list

{{{DONE find "${WEB_APP_DIR}/archive/apache.org/announce/" -type f | sort | xargs -n 1 -I {} ~noel/update-list 'announce@apache.org' {} ["DONE"] find "${WEB_APP_DIR}/archive/apache.org/mirrors/" -type f | sort | xargs -n 1 -I {} ~noel/update-list 'mirrors@apache.org' {} }}}

• Synch the current month
  • Copy current month (on nagoya as qmlist, in the right directory)

{{{DONE wget apache.org/mail/announce/200308 ["DONE"] wget apache.org/mail/mirrors/200308 }}}

• Subscribe lists (on daedalus)

{{{DONE ezmlm-sub /home/apmail/lists/apache.org/announce/ qmlist-apache-archive-announce@nagoya.apache.org ["DONE"] ezmlm-sub /home/apmail/lists/apache.org/mirrors/ qmlist-apache-archive-mirrors@nagoya.apache.org }}}

• Index current month (on nagoya as regular user)

{{{DONE ~noel/update-list 'announce@apache.org' "${WEB_APP_DIR}/archive/apache.org/announce/200308" ["DONE"] ~noel/update-list 'mirrors@apache.org' "${WEB_APP_DIR}/archive/apache.org/mirrors/200308" }}}