How to optimise concurrent access to an index
It should be possible for multiple lucene processes to share an index as long as they don't both try to write a the same time. I've (KevinBurton) been able to verify this by testing a full index merge on a production system while still doing full text searches during the merge.
Things of course were slower but not by much.
The trick is not to create another IndexWriter while the other process is writing to the index.
Also see LockObtainTimedOut for more info on how to debug the inability to obtain a lock for an index.