Lucene Indexing: Clearing Confusion

  • MHB
  • Thread starter shivajikobardan
  • Start date
  • Tags
    Confusion
In summary, the conversation discusses the steps of indexing in Lucene, specifically the role of the IndexWriter object in the process. The first and third steps are not clearly explained in the diagram provided and may differ from other sources. The IndexWriter is responsible for incorporating documents into the index and can also add, update, and delete indexes.
  • #1
shivajikobardan
674
54
https://lh5.googleusercontent.com/47guV-L3yY2ZevuNEwk1wC9t9rJjQw0bXNHug16ah2EQ2XyLTAzqrBZcDMEwzFSd1mR_jFDTOFyG1GVHT8p1G4tPPRkRtqtOcOGXTb3UrildRHMayRznHaFQD9RdCdjeuEvyM-FkvQ_U3GLBHGVgkFY
These are the steps of indexing in Lucene given in our syllabus-:
https://lh4.googleusercontent.com/Qliby1unxHTU0vAwycWZzy563XdxcwUT4UAyA6Xf1ydKQAwSfKwqexdDFFc0CBZb9kSSvRKXEFoyKQ4cYn9K2EgEDRnWTiYFCDlqZ4VCAw9CWvgvcI9cOJo055PCJhyFTBJckNhtLi-eAMwM7q8JoUU

https://lh3.googleusercontent.com/X9geIBuCERbSadCkshBekbjvl4GAqGvHppgCayGcOBvaIkpIX1Jy5jyFSmmp39ANIyg3cq0tYWrYTxl1RNlOUfbHFAcNy5CJLxfWGve6DpjeXXekNwTl3P64zQ1_6dojvdo4-Z8aTFn-EZ51CqOlLfE
I understand the second step clearly. But I don't understand the first and third step. It's not mentioned clearly in this figure imo. Can you clear my confusion? Plus the sources that I refer don't even mention it like this, they explain it differently. I'm not sure from where this is copied from.
What are we doing in first vs third step as written in that figure text?
Why was indexwriter created first and not used later? Because according to my information that I've collected, you can also use indexwriter to add/remove/update indexes.

I've a good feeling that all of this information is incorrect but this is what's written in my teacher's notes so I'm not 100% sure of it. And even if it's wrong, they'd expect us to write the same thing in exam, so I've to learn it.
 
Technology news on Phys.org
  • #2
shivajikobardan said:
But I don't understand the first and third step.
Are you talking about the first 3 steps? The "build index" step and "IndexWriter object" step?

The top diagram is simply a architecture diagram, not a flow diagram. It's show the role in the search/index process that Lucene fills.

The 3 steps at the top, with the IndexWriter, represent the lower let portion of that architecture. This is where the documents are being incorporated in the Lucene index, so that they can alter be searched (that's the upper right portion of the diagram).

The IndexWriter is simply the mechanic to route documents into the index. The IndexWriter handles the lexing and parsing of the documents to prepare them for the actual index. And, yes, the IndexWriter can also add/update/delete indexes.

Let me know if you have other questions, I know this response is a bit late. So it may no longer be timely.
 

FAQ: Lucene Indexing: Clearing Confusion

What is Lucene indexing?

Lucene indexing is a process used in information retrieval to create a searchable index from a collection of documents. It is an open-source search engine library that allows for efficient and accurate indexing and searching of large amounts of text data.

How does Lucene indexing work?

Lucene indexing works by analyzing the text content of a document and breaking it down into smaller units, such as words or phrases, known as tokens. These tokens are then stored in an inverted index, which maps each token to the documents that contain it. This allows for fast retrieval of documents that contain specific words or phrases.

What is the purpose of Lucene indexing?

The main purpose of Lucene indexing is to enable efficient and accurate searching of large collections of documents. It helps users to quickly find relevant information by creating a searchable index that maps words or phrases to the documents that contain them. This is especially useful for applications such as search engines, e-commerce sites, and content management systems.

How is Lucene indexing different from other indexing methods?

Lucene indexing is different from other indexing methods in that it is specifically designed for text-based data. It uses advanced algorithms and data structures to efficiently index and search large amounts of text, making it a popular choice for search engine and information retrieval applications. Other indexing methods may be better suited for different types of data, such as numerical or spatial data.

Is Lucene indexing suitable for all types of documents?

Yes, Lucene indexing is suitable for all types of documents, including text, HTML, XML, PDF, and Microsoft Office documents. It can also handle a variety of languages and character encodings. However, it may require some customization or additional tools for more complex document types, such as images or videos.

Back
Top