URL Aliases

mnoGoSearch has an alias support making it possible to index sites taking information from another location. For example, if you index local web server, it is possible to take pages directly from disk without involving your web server in indexing process. Another example is building of search engine for primary site and using its mirror while indexing. There are several ways of using aliases.

Specifying aliases

To specify an alias go to Access tab in Server settings.

E.g. you wish to index http://search.mnogo.ru/ using nearest German mirror http://www.gstammw.de/mirrors/mnoGoSearch/. Enter the following URLs in corresponding fields:

Server: http://search.mnogo.ru/

Alias: http://search.mnogo.ru/ http://www.gstammw.de/mirrors/mnoGoSearch/

search.exe will display URLs from master site http://search.mnogo.ru/ but indexer will take corresponding page from mirror site http://www.gstammw.de/mirrors/mnoGoSearch/.

Another example. If you want to index everything in udm.net domain and one of servers, for example http://home.udm.net/ is stored on local machine in /home/httpd/htdocs/ directory. Set type Realm reg. exp. in Server settings Access tab, and the following URLs:

Server http://*.udm.net/

Alias http://home.udm.net/ file:/home/httpd/htdocs/

Indexer will take home.udm.net from local disk and index other sites using HTTP.

Using alias in Server command

You may specify location used by indexer as an optional argument for Server field:

Server http://home.udm.net/ file:/home/httpd/htdocs/

Using alias in Realm match type

Aliases in Realm command is a very powerful feature based on regular expressions. The idea of aliases in Realm implementation is similar to how PHP preg_replace() function works. Aliases in Realm match type work only if reg.ex. match type is used and does not work with string match type. To use this feature, select Realm reg.exp in Type menu of Server settings Access tab.

Indexer searches URL for matches to URL_pattern, entered into Server field and build an URL alias using alias_pattern, entered into Alias field. alias_pattern may contain references of the form $n. Where n is a number in the range of 0-9. Every such reference will be replaced by text captured by the n'th parenthesized pattern. $0 refers to text matched by the whole pattern. Opening parentheses are counted from left to right (starting from 1) to obtain the number of the capturing subpattern.

For example: your company hosts several hundreds users with their domains in the form of www.username.yourname.com. Every user's site is stored on disk in "htdocs" under user's home directory: /home/username/htdocs/.

You may write this command into Server field (note that dot '.' character has a special meaning in regular expressions and must be escaped with '\' sign when dot is used in usual meaning):

(http://www\.)(.*)(\.yourname\.com/)(.*) file:/home/$2/htdocs/$4

E.g. indexer processes the following page: http://www.john.yourname.com/news/index.html

It will build patterns from $0 to $4:

Table 10.1.

Then indexer will compose alias using $2 and $4 patterns:

file:/home/john/htdocs/news/index.html

and will use the result as document location to fetch it.