ಮ್ಯಾಂಟಿಕೋರ್‌ನೊಂದಿಗೆ ಪೈಥಾನ್ ಅಪ್ಲಿಕೇಶನ್‌ಗಾಗಿ ಸ್ಕೇಲೆಬಲ್ ಸರ್ಚ್ ಇಂಜಿನ್ ಅನ್ನು ಹೇಗೆ ನಿರ್ಮಿಸುವುದು

One of the challenges with using Manticore directly from your code or from inside Python’s REPL (or any other language) is to write properly structured queries. This requires that you know about writing parsers, which can be a little overwhelming for newbies in programming but not necessarily for our purposes here: we are going to stick to two basic types of queries: simple keyword searches and exact phrase matches. To that end, let’s take a look at an example query and see how it should be formatted, before moving on to create the parser itself. For this section we are going to use HTTP requests via cURL :

If you want more information about cURL , take a look at this guide.

First, extract the query string from the request, which is done by writing some code similar to this:

Then you need to break it into tokens, which can be achieved with something like this:

Adding Quotes and Spaces as Supported Token Types adds support for exact phrase matches as well (see Chapter 3 of the “ Search Full Text of PDF and HTML in Manticore ” guide for more details). The question remains how to check if a token is an exact phrase match or not. This can be done by checking its distance from other tokens in your dictionary. For example, let’s say we have a very simple vocabulary consisting of only two words (a and b):

Now, you can check if token is an exact phrase match by measuring its distance from all other tokens in the index. If it is greater than 1, then we know it’s not a phrase match:

In this article we are going to implement support for exact phrase matches ourselves. However, I highly recommend that you use Manticore’s built-in support for this instead as described in Chapter 7 of the “ Search Full Text of PDF and HTML in Manticore ” guide. You will also learn how to configure your analyzer to achieve specific results such as stop words removal and stemming – which are important optimization techniques when building search engines. We can start by defining the code to build a query parser as follows:

You’ll notice that I used the class name QueryParser instead of Tokenizer because this one actually parses text into tokens (with help from our vocabulary) and then builds the search query. Let’s go through each method in turn, starting with tokenize() :

Tokens can be built from our vocabulary or from individual characters by using the function get_token . The latter is useful for exact phrase matches which we are going to look at now. If you want more information about how to implement different types of queries such as prefix matching, take a look at Chapter 4 of “ Search Full Text of PDF and HTML in Manticore ” guide. For now, let’s look at exact phrase matches:

In this case we are using a simple regular expression to match the tokens we need and then either replace them with themselves or add quotes depending on if they have whitespace around them. In other words, it replaces single word keywords with itself (e.g. ‘cat’) but adds quotes around exact phrase matches that include spaces (e.g. “cat in the hat”). This is an extremely simplistic implementation of what will be a very important optimization in full-text search engines and I highly recommend you read more about it as well as implement proper query parsers for your own needs before moving on!

You can now start writing queries against your index again:

And you should see the results you expect! Great job!