Goal: Setup indexing for incubator lists
Status:
DONE announce
DONE general
DONE geronimo-dev
DONE genonimo-cvs -- start indexing after there is an mbox file
DONE projects
DONE cvs
- On nagoya as qmlist user
- Make new mbox archive area(s)
{{{DONE mkdir -p /opt/ezmlm/archive/incubator.apache.org/announce ["DONE"] mkdir -p /opt/ezmlm/archive/incubator.apache.org/general ["DONE"] mkdir -p /opt/ezmlm/archive/incubator.apache.org/cvs ["DONE"] mkdir -p /opt/ezmlm/archive/incubator.apache.org/projects ["DONE"] mkdir -p /opt/ezmlm/archive/incubator.apache.org/geronimo-dev ["DONE"] mkdir -p /opt/ezmlm/archive/incubator.apache.org/geronimo-cvs }}}
- Create .qmail file
["DONE"] echo '|$HOME/sbin/archive -u "$HOME/archive/incubator.apache.org/$DEFAULT/%Y%m"' > ~qmlist/.qmail-incubator-archive-default
- Copy over initial mbox file(s):
{{{ *** daedalus:/www/tlp.apache.org/mail/list has the mbox files in compressed format. They need to be uncompressed before running eyebrowse.
["DONE"] wget incubator.apache.org/mail/general/200210.gz ["DONE"] wget incubator.apache.org/mail/general/200211.gz ["DONE"] wget incubator.apache.org/mail/general/200212.gz ["DONE"] wget incubator.apache.org/mail/general/200301.gz ["DONE"] wget incubator.apache.org/mail/general/200302.gz ["DONE"] wget incubator.apache.org/mail/general/200303.gz ["DONE"] wget incubator.apache.org/mail/general/200304.gz ["DONE"] wget incubator.apache.org/mail/general/200305.gz ["DONE"] wget incubator.apache.org/mail/general/200306.gz ["DONE"] wget incubator.apache.org/mail/general/200307.gz ["DONE"] gunzip *.gz
["DONE"] wget incubator.apache.org/mail/announce/200302.gz ["DONE"] wget incubator.apache.org/mail/announce/200303.gz ["DONE"] wget incubator.apache.org/mail/announce/200304.gz ["DONE"] wget incubator.apache.org/mail/announce/200305.gz ["DONE"] wget incubator.apache.org/mail/announce/200306.gz ["DONE"] wget incubator.apache.org/mail/announce/200307.gz ["DONE"] gunzip *.gz
["DONE"] wget incubator.apache.org/mail/cvs/200210.gz ["DONE"] wget incubator.apache.org/mail/cvs/200211.gz ["DONE"] wget incubator.apache.org/mail/cvs/200212.gz ["DONE"] wget incubator.apache.org/mail/cvs/200301.gz ["DONE"] wget incubator.apache.org/mail/cvs/200302.gz ["DONE"] wget incubator.apache.org/mail/cvs/200303.gz ["DONE"] wget incubator.apache.org/mail/cvs/200304.gz ["DONE"] wget incubator.apache.org/mail/cvs/200305.gz ["DONE"] wget incubator.apache.org/mail/cvs/200306.gz ["DONE"] wget incubator.apache.org/mail/cvs/200307.gz ["DONE"] gunzip *.gz
["DONE"] wget incubator.apache.org/mail/projects/200302.gz ["DONE"] wget incubator.apache.org/mail/projects/200303.gz ["DONE"] wget incubator.apache.org/mail/projects/200304.gz ["DONE"] wget incubator.apache.org/mail/projects/200305.gz ["DONE"] wget incubator.apache.org/mail/projects/200306.gz ["DONE"] wget incubator.apache.org/mail/projects/200307.gz ["DONE"] gunzip *.gz }}}
- On nagoya as regular user
- Register new lists
{{{ export WEB_APP_DIR='/opt/tomcat/webapps/eyebrowse'
- export EYEBROWSE="${WEB_APP_DIR}/WEB-INF/bin/eyebrowse --props=${WEB_APP_DIR}/WEB-INF/classes/eyebrowse.properties"
["DONE"] ${EYEBROWSE} --command=addlist --listname='announce@incubator.apache.org' --descr='Apache Incubator Announcements' --indexdir="${WEB_APP_DIR}/index/incubator.apache.org/announce" ["DONE"] ${EYEBROWSE} --command=addlist --listname='cvs@incubator.apache.org' --descr='Apache Incubator CVS Records' --indexdir="${WEB_APP_DIR}/index/incubator.apache.org/cvs" ["DONE"] ${EYEBROWSE} --command=addlist --listname='general@incubator.apache.org' --descr='Apache Incubator General Discussion' --indexdir="${WEB_APP_DIR}/index/incubator.apache.org/general" ["DONE"] ${EYEBROWSE} --command=addlist --listname='projects@incubator.apache.org' --descr='Apache Incubator Projects' --indexdir="${WEB_APP_DIR}/index/incubator.apache.org/projects" ["DONE"] ${EYEBROWSE} --command=addlist --listname='geronimo-dev@incubator.apache.org' --descr='Apache Incubator Geronimo Discussion' --indexdir="${WEB_APP_DIR}/index/incubator.apache.org/geronimo-dev" ["DONE"] ${EYEBROWSE} --command=addlist --listname='geronimo-cvs@incubator.apache.org' --descr='Apache Incubator Geronimo CVS Records' --indexdir="${WEB_APP_DIR}/index/incubator.apache.org/geronimo-cvs" }}}
- Add initial mbox file to the database, update the database and Lucene indices:
- ~noel/update-list implements addlistfile, updatelist, and updateindex in order
- Repeat for each initial mbox for each list
{{{DONE find "${WEB_APP_DIR}/archive/incubator.apache.org/announce/" -type f | sort | xargs -n 1 -I {} ~noel/update-list 'announce@incubator.apache.org' {} ["DONE"] find "${WEB_APP_DIR}/archive/incubator.apache.org/cvs/" -type f | sort | xargs -n 1 -I {} ~noel/update-list 'cvs@incubator.apache.org' {} ["DONE"] find "${WEB_APP_DIR}/archive/incubator.apache.org/general/" -type f | sort | xargs -n 1 -I {} ~noel/update-list 'general@incubator.apache.org' {} ["DONE"] find "${WEB_APP_DIR}/archive/incubator.apache.org/projects/" -type f | sort | xargs -n 1 -I {} ~noel/update-list 'projects@incubator.apache.org' {} ["DONE"] ${EYEBROWSE} --command=addlistfile --listname='geronimo-dev@incubator.apache.org' --listfile="${WEB_APP_DIR}/archive/incubator.apache.org/geronimo-dev/200308" ["DONE"] ${EYEBROWSE} --command=addlistfile --listname='geronimo-cvs@incubator.apache.org' --listfile="${WEB_APP_DIR}/archive/incubator.apache.org/geronimo-cvs/200308" }}}
- Synch the current month
- Copy current month (on nagoya as qmlist, in the right directory)
{{{DONE wget incubator.apache.org/mail/general/200308 ["DONE"] wget incubator.apache.org/mail/announce/200308 ["DONE"] wget incubator.apache.org/mail/cvs/200308 ["DONE"] wget incubator.apache.org/mail/projects/200308 }}}
- Subscribe lists (on daedalus)
{{{DONE ezmlm-sub /home/apmail/lists/incubator.apache.org/announce/ qmlist-incubator-archive-announce@nagoya.apache.org ["DONE"] ezmlm-sub /home/apmail/lists/incubator.apache.org/general/ qmlist-incubator-archive-general@nagoya.apache.org ["DONE"] ezmlm-sub /home/apmail/lists/incubator.apache.org/cvs/ qmlist-incubator-archive-cvs@nagoya.apache.org ["DONE"] ezmlm-sub /home/apmail/lists/incubator.apache.org/projects/ qmlist-incubator-archive-projects@nagoya.apache.org ["DONE"] ezmlm-sub /home/apmail/lists/incubator.apache.org/geronimo-dev/ qmlist-incubator-archive-geronimo-dev@nagoya.apache.org ["DONE"] ezmlm-sub /home/apmail/lists/incubator.apache.org/geronimo-cvs/ qmlist-incubator-archive-geronimo-cvs@nagoya.apache.org }}}
- Index current month (on nagoya as regular user)
{{{DONE ~noel/update-list 'announce@incubator.apache.org' "${WEB_APP_DIR}/archive/incubator.apache.org/announce/200308" ["DONE"] ~noel/update-list 'general@incubator.apache.org' "${WEB_APP_DIR}/archive/incubator.apache.org/general/200308" ["DONE"] ~noel/update-list 'cvs@incubator.apache.org' "${WEB_APP_DIR}/archive/incubator.apache.org/cvs/200308" ["DONE"] ~noel/update-list 'projects@incubator.apache.org' "${WEB_APP_DIR}/archive/incubator.apache.org/projects/200308" }}}