Tuesday, October 20, 2009

Released BrokenCompass

So finally after much of ifs and buts and all hesitation have released a version of BrokenCompass.. Full text search engine built using ruby on rails and serving ruby on rails app.

Some of the features which are available as of now in this first release are..
  1. Able to work with all databases for which ruby in rails have no issues(examples shown in links uses mysql)
  2. Availability of extended_mode which allows to search in any particular attribute(from set of attributes which is used while in index creation)
  3. In extended_mode queries can have logical "&"(pronounced as 'and') and "|"(pronounced as 'or') between them(Not operator is not available still, but working out to release in next version).
  4. One can assign weight to set of attributes depending upon requirements and in response get weighted result and also sorted in descending order of weight of record.
  5. By default 20 records are returned as query response(which can be altered)
  6. look_in_brokencompass static method is attached to rails model to get inference of instantaneous parameter set. And to find using brokencompass static method is find_with_brokencompass("searchterm", :brokencompass = {}) and many other..

Here are few links(if interested, will be helpfull)
BrokenCompass: How to go about it
BrokenCompass - Install instructions

Monday, October 19, 2009

BrokenCompass: How to go about it

Once you are done with installation and index creation for brokencompass, can play around with the search capabilities of it..

Point 01. ruby script/console
==> Output
Adding BrokenCompass Version: 1.0
Loading index... [broken_compass/advertisment.index]

Point 02. Searching in all fields, i.e not using extended_mode
Advertisment.look_in_brokencompass("train")
==> Returns an inference hash like this
{
# Time elapsed in the execution(in seconds)
:time_elapsed=>0.00617599487304688,

# Field(s) which is/are available for indexing
:fields=>["location", "title", "description"],

# Weight given to different attributes to ranks result items
:column_weight=>{:description=>10, :location=>10, :title=>10},

# Count of total records found
:total_record_count=>44,

# Boolean field to unload indexes from memory once query is fired
:eager_unload=>false,

# Boolean field for extended_mode, set to true when using attribute(column) level searches
:extended_mode=>false,

# Boolean field for PLURAL forms of search word
:plural_forms=>true,

# Result set with weighted scores
:weighted_records=>[[14844, 20], [17445, 20], ... , [15148, 10]]}

# Limit to result set
:limit=>20,

# Offset to result set
:offset=>0,
}


Point 03. Searching in extended mode i.e specifically in some attribute(from the attributes used in sql query for index creation)
Advertisment.look_in_brokencompass("'job'@title", :extended_mode => true)
==> Returns an inference hash same as above
{
:column_weight=>{:description=>10, :location=>10, :title=>10},
:total_record_count=>1419,
:time_elapsed=>0.0523371696472168,
:fields=>["location", "title", "description"],
:eager_unload=>false,
:limit=>20,
:extended_mode=>true,
:offset=>0,
:plural_forms=>true,
:weighted_records=>[[14607, 20], [17685, 20], ...... , [13727, 10]]
}


Point 04. Searching in extended mode and also using other possible options
Advertisment.look_in_brokencompass("'job'@title", :extended_mode => true, :offset => 4, :limit => 5, :column_weight => {:title => 40, :description => 20})
==> Returns an inference hash same as above
{
:column_weight=>{:description=>20, :location=>10, :title=>40},
:total_record_count=>1419,
:time_elapsed=>0.0521509647369385,
:fields=>["location", "title", "description"],
:eager_unload=>false,
:limit=>5,
:extended_mode=>true,
:offset=>4,
:plural_forms=>true,
:weighted_records=>[[17947, 80], [17946, 80], [9597, 80], [12556, 80], [25975, 80]]
}

Point 05. To retrieve data results from above mentioned conditions and critriea
Advertisment.find_with_brokencompass("'job'@title", :brokencompass => {:extended_mode => true, :offset => 4, :limit => 5, :column_weight => {:title => 40, :description => 20}})
==> Returns an result-set array of type Advertisment

Point 06. To user "and" & "or" operators in extended mode
Advertisment.look_in_brokencompass("'job'@title | 'career'@description", :extended_mode => true, :offset => 4, :limit => 5, :column_weight => {:title => 40, :description => 20})
==> Returns an result-set array of type Advertisment
Advertisment.look_in_brokencompass("'job'@title & 'career'@description", :extended_mode => true, :offset => 4, :limit => 5, :column_weight => {:title => 40, :description => 20})
==> Returns an result-set array of type Advertisment

In case of any further queries please contact me at praveen[dot]kumar[dot]sinha[at]gmail[dot]com

BrokenCompass - Install instructions

BrokenCompass up and running in flat 5 mins

Step: 01
Install BrokenCompass plugin
ruby script/plugin install http://brokencompass.googlecode.com/svn/trunk
from inside of rails app root directory.

Step: 02
Add BrokenCompass configuration yml file
Create file
config/brokencompass.yml

Add content like
#brokencompass.yml............. STARTS
index-sources:
advertisment:
sql: "select id,title,description,location from advertisments"
table_name: advertisments

query:
sql: "select id,content,dynamic_fields from queries"
table_name: queries

# Database connection adapter
connection_adapter:
adapter: "mysql"
host: "localhost"
username: "username"
password: "password"
database: "myapp_development"

#brokencompass.yml............. ENDS

Note:
0. Nothing is optional in the yml(everything is mandatory)
1. Add as many items in index-sources as you want, but every item name should match model names(for which they will be used).
2. Make sure to include the Id field so that index created can be mapped to record identifier.
3. Provide the connection adapter details in connection_adapter section(so as to answer, where to read data from)
4. If you are not minding the spaces in yml(then you have messed up the configuration, so don't try to remove spaces :) from yml, wanna know more about yml? )


Step: 03
Run rake task to create indexes for the first time
rake brokencompass:create_index
from inside of rails app root directory.

Step: 04 (This is required, when one is re-creating the indexes from scratch)
Run rake task to re-create indexes
rake brokencompass:index
from inside of rails app root directory.

Step: 05 (To delete any of the indexes so created, delete corresponding "xxxx.index" file from broken_compass
from inside of rails app root directory.

Friday, October 9, 2009

BrokenCompass

New in-memory full text search engine in ruby and in rails "BrokenCompass".
Coming soon....
keep an eye on this space..

Honda Civic & A/C problems.

Hello Friends, Got a post again on to Honda Civic (The good old favorite commuter of mine). This car has been doing great except for so...