吕克让的资料库 - lhelper's repository

克己服人,礼智谦让!
Weblcome to lhelper's repository!

Tuesday, July 04, 2006

 

JavaCC based Simple-QueryParser For Lucene

修改了Lucene 自带的QueryParser,
屏蔽了Token Definitions 中的部分内容,去掉了对:
RangeQuery、
WildcardQuery、
PrefixQuery、
( Query )、
boost、
slop
等的直接支持——以上几项可以改为用Filter 来实现

加入了对不完整表达式的容错处理,减少ParserException 的发生:
Java AND AND Lucene
Java AND "Lucene
Java AND Lucene"
Java AND
AND Lucene
title:
:Java

加入validFieldSet,用于限制可以出现在":" 左侧的有效field,防止:
tom said: "hi everyboy"
主持人:"大家好"
等被解析成错误的Query



如果Analyzer 过滤了所有term 的话,则让parser() 方法返回null,如:
+AND a
+OR NOT the was will
parser() 方法返回null 的情况下,可以仿照google 将请求转向到另外一个页面。


当整个Query 中的所有BooleanClause 都是 prohibited (Pure NOT Query) 的情况下,可以选择:
将第一个clause 的prohibited 属性至为false
在当前clauses 的基础上再加一个MatchAllDocsQuery (http://lucene.apache.org/java/docs/api/org/apache/lucene/search/MatchAllDocsQuery.html)
直接返回pure-NOT-query



屏蔽了部分内容之后的 Token Definitions (/// 为注释部分):


/* ***************** */
/* Token Definitions */
/* ***************** */


<*> TOKEN : {
<#_NUM_CHAR: ["0"-"9"] >
<#_ESCAPED_CHAR: "\\" [ "\\", "+", "-", "!", "(", ")", ":", "^",
"[", "]", "\"", "{", "}", "~", "*", "?" ] >
<#_TERM_START_CHAR: ( ~[ " ", "\t", "\n", "\r", "+", "-", "!", "(", ")", ":", "^",
"[", "]", "\"", "{", "}", "~", "*", "?" ]
<_ESCAPED_CHAR> ) >
<#_TERM_CHAR: ( <_TERM_START_CHAR> <_ESCAPED_CHAR> "-" "+" ) >
/// <#_WHITESPACE: ( " " "\t" "\n" "\r") >
<#_SKIP_CHAR: (" " "\t" "\n" "\r" "(" ")" "{" "}" "[" "]" "^" "~" "*" "?") >
}



/// <DEFAULT, RangeIn, RangeEx> SKIP : {
/// <<_WHITESPACE>>
/// }
<DEFAULT> SKIP : {
<<_SKIP_CHAR>>
}


// OG: to support prefix queries:
// http://nagoya.apache.org/bugzilla/show_bug.cgi?id=12137
// Change from:
// <WILDTERM: <_TERM_START_CHAR>
// (<_TERM_CHAR> ( [ "*", "?" ] ))* >
// To:
//
// <WILDTERM: (<_TERM_CHAR> ( [ "*", "?" ] ))* >



<DEFAULT> TOKEN : {
<AND: ("AND" "&&") >
<OR: ("OR" "") >
<NOT: ("NOT" "!") >
<PLUS: "+" >
<MINUS: "-" >
/// <LPAREN: "(" >
/// <RPAREN: ")" >
<COLON: ":" >
/// <CARAT: "^" > : Boost
<BILATERAL_QUOTED: "\"" (~["\""])+ "\"">
<UNILATERAL_QUOTED: "\"" (~["\""])+>
<QUOTE_AT_THE_END: "\"">
<NUMBER: (<_NUM_CHAR>)+ ( "." (<_NUM_CHAR>)+ )? >
<TERM: <_TERM_START_CHAR> (<_TERM_CHAR>)* >
/// <FUZZY: "~" >
/// <SLOP: "~" (<_NUM_CHAR>)+ >
/// <PREFIXTERM: <_TERM_START_CHAR> (<_TERM_CHAR>)* "*" >
/// <WILDTERM: <_TERM_START_CHAR>
/// (<_TERM_CHAR> ( [ "*", "?" ] ))* >
/// <RANGEIN_START: "[" > : RangeIn
/// <RANGEEX_START: "{" > : RangeEx
}



/// <Boost> TOKEN : {
/// <NUMBER: (<_NUM_CHAR>)+ ( "." (<_NUM_CHAR>)+ )? > : DEFAULT
/// }


/// <RangeIn> TOKEN : {
/// <RANGEIN_TO: "TO">
/// <RANGEIN_END: "]"> : DEFAULT
/// <RANGEIN_QUOTED: "\"" (~["\""])+ "\"">
/// <RANGEIN_GOOP: (~[ " ", "]" ])+ >
/// }



/// <RangeEx> TOKEN : {
/// <RANGEEX_TO: "TO">
/// <RANGEEX_END: "}"> : DEFAULT
/// <RANGEEX_QUOTED: "\"" (~["\""])+ "\"">
/// <RANGEEX_GOOP: (~[ " ", "}" ])+ >
/// }


Comments: Post a Comment



<< Home

Archives

June 2004   November 2005   July 2006   August 2006  

This page is powered by Blogger. Isn't yours?