Solr

Overview of Solr 1.4

Get Started. It's Free
or sign up with your email address
Rocket clouds
Solr by Mind Map: Solr

1. Indexing

1.1. CSV, XML, JDBC

1.2. Streaming data from file system

1.3. Other document formats

1.3.1. Solr Cell

1.3.1.1. Thin Layer

1.3.1.2. Uses Apache Tika

1.3.1.2.1. Delegates to 3rd Party Libraries

1.3.1.2.2. Generates SAX events

1.3.1.2.3. Media Files - mp3 etc,

1.4. POJOs

1.4.1. Embedded Solar ?

2. Text Analysis

2.1. Extracting terms

2.1.1. Documents

2.1.2. Queries

2.1.3. Analyzer Chains for each

2.2. Tokenizer

2.3. Stemming

2.3.1. Reducing variants to one representation

2.4. Synonyms

2.4.1. Query Time

2.5. Index Time

2.5.1. Stems are indexed

2.5.2. Better scores for rare terms

2.5.3. Adding synonyms requires re-indexing

2.6. Stop Words

2.7. Phonetic Search

2.8. Wild Card

2.8.1. N-Gram

2.8.1.1. Minimum, Maximum

2.8.1.2. To, Ton, Toni, Tonig ...

2.8.1.3. Resource Intensive

2.8.2. Only Trailing

3. Lucene

3.1. Library, not Server

3.2. Not a Web Crawler

3.3. Inverted Index

3.4. Scoring Algortihm

3.5. Text Analyzers

3.6. Query Syntax

3.7. Highlighter

4. Other Features

4.1. Serverization of Solr

4.2. HTTP based API

4.3. Caching of results

4.4. Spell Checker

4.5. More Like This

4.6. Highlighting

5. Schema Design

5.1. Single vs. Multiple Index

5.1.1. Multiple entities

5.1.2. Pros

5.1.2.1. Easier to search across entities

5.1.2.2. Easy to maintain

5.1.3. Cons

5.1.3.1. Namespace Collision

5.1.3.1.1. Add prefix to each field

5.1.3.2. Lower Quality for rare terms

5.1.3.2.1. Term may not be rare across entities

5.1.3.3. Slower performance

5.1.3.3.1. Fuzzy, prefix, Wildcard Searches

5.1.3.3.2. Frequent commits to a single entity invalidates entire cache

5.2. Design

5.2.1. Determine common searches

5.2.2. Identify the key entities

5.2.3. Denormalize related entities

5.2.3.1. Use multi-valued felds

5.2.4. Omit fields that are only used to display

5.2.4.1. Optional - Performance Hit

6. Searching

6.1. Query Expressions

6.1.1. + Mandatory

6.1.2. - Prohibited

6.1.3. Optional

6.1.4. fileldname:xxx Field Qualifier

6.1.5. " xxx yyy" Phrases

6.1.6. "xxx yyy"~3 Term Proximity

6.1.7. Range Queries

6.2. Boosting

6.2.1. < 1 Reduce

6.2.2. > 1 Increase

6.2.3. fieldname:xxx ^ 2

6.2.4. Phrase Boosting

6.2.4.1. "Billy Joel" is higher

6.2.5. Boost Queries

6.2.5.1. Recent albums are higher

6.3. Filtering

6.3.1. Limit what the user sees

6.3.2. fq:...

6.3.3. Filter Queries are cached

6.3.4. Does not affect the score

6.4. Scoring

6.4.1. Relative, not Absolute measure

6.4.2. Term Frequency - tf

6.4.2.1. More frequent higher score

6.4.3. Inverse Document Frequency - idf

6.4.3.1. Rare term higher score

6.4.4. Coordination Factor

6.4.4.1. More terms match higher score

6.4.5. Field Length

6.4.5.1. Shorter the matched term higher score

6.4.5.2. Smashing > Smashing Pumpkin

6.5. Faceting

6.5.1. Text

6.5.2. Dates

6.5.3. Queries

6.5.3.1. Create Ranges

6.5.3.1.1. [0-2], [2-4], [>4] ..

6.5.4. Sort by Count

6.5.5. Limit to x values

6.5.6. Power auto suggest

7. Deployment

7.1. Search Handlers

7.1.1. For different user categories

7.1.1.1. Paying / Non-Paying

7.1.1.2. Regular / Admin

7.2. Cores

7.2.1. Rebuilding Indexes

7.2.1.1. Bulk updates of separate index

7.2.1.2. Searchers not impacted

7.2.1.3. SWAP to activate the new index

7.2.2. Testing Configuration Changes

7.2.3. Operations

7.2.3.1. SWAP

7.2.3.2. MERGE

7.2.3.3. RENAME

7.3. Security

7.3.1. Standard - behind the firewall

7.3.2. WAR file - web.xml controls

7.3.3. AJAX requests

7.3.3.1. Search Handlers

7.3.4. Document Access

7.3.4.1. Use Faceting

7.3.4.1.1. Multi valued Facet for Role

7.3.4.1.2. Index document with appropriate Role values

7.3.4.1.3. Add facet to queries

8. Integration

8.1. Embedded Solr

8.1.1. Streaming Local Content

8.1.2. Upgrading existing Lucene to Solr

8.2. JSON Support

8.2.1. Return results to Javascript

9. Scaling

9.1. # Users

9.1.1. Multiple Servers

9.1.1.1. Master Slave support

9.1.1.1.1. Index into the Master

9.1.1.1.2. Solr Replicates

9.1.1.1.3. External Load Balancer

9.1.1.2. New node

9.1.2. Single Server

9.1.2.1. Caching

9.1.2.1.1. Leverage HTTP Caching

9.1.2.1.2. External Cache (e.g. Squid)

9.1.2.1.3. Optimization

9.1.2.2. Schema Design

9.1.2.2.1. Index only essential fields

9.1.2.2.2. Store only essential fields

9.1.2.3. Indexing Strategy

9.1.2.3.1. Tune the Batch Size

9.1.2.4. Enable Term Vectors

9.1.2.4.1. List of terms extracted from a field

9.1.2.4.2. Faster Retrieval - precomputation

9.1.2.4.3. More Like This

9.1.2.4.4. Highlighting

9.1.2.5. Shingling

9.1.2.5.1. Faster Phrase Search

9.1.2.5.2. "quick brown", "lazy dog" ...

9.1.2.5.3. Fields where phrase search is common

9.2. Amount of Data

9.2.1. Sharding Support

9.2.1.1. Solr aggregates results from shards

9.2.1.2. Need to manage document to shard assgnment