|
ÀÁÐÀÊÀÄÀÁÐÀ (Òîæå ñàìîå íî â ÷èòàåìîì âèäå)
Ífcnjzofz hf,jnf
zdkztncz jnxtnjv j, exfcnbb d rjyathtywbb ÐÎÌÈÏ-2008. Ãkfdyjq wtkm.
hf,jns ,skf fghj,fwbz vtnjljd hfcxtnf htktdfynyjcnb ljrevtynf pfghjce
ghb gjbcrt gj Web-cnhfybwfv b rjkktrwbb yjhvfnbdys[ ljrevtynjd. Âdtltybt MnoGoSearch
zdkztncz cdj,jlyj hfcghjcnhfyztvjq gjbcrjdjq cbcntvjq, hf,jnf.otq d
jgthfwbjyys[ cbcntvf[ ctvtqcndf Unix, ghtlyfpyfxtyyjq lkz jhufybpfwbb
gjbcrf yf jlyjv bkb vyjub[ Web-cthdthf[. Ïthdfz dthcbz mnoGoSearch ,skf
dsgeotyf d yjz,ht 1998 gjl yfpdfybtv UDMSearch. Â jrnz,ht 2001 ujlf
gjzdbkbcm rjvvthxtcrbt vjlbabrfwbb cbcntvs, htfkbpjdfyyst lkz
jgthfwbjyys[ cbcntv Windows. Ïjcktlybt dthcbb cbcntvs vj;yj yfqnb yf
cfqnt http://www.mnogosearch.org/. 1. Êhfnrjt jgbcfybt cbcntvs mnoGoSearch cjcnjbn bp lde[ xfcntq. Ïthdfz xfcnm - byltrcbhe.obq vt[fybpv (indexer). Indexer ghj,tuftn
gj ccskrfv b cj[hfyztn d ,fpt lfyys[ byajhvfwb. j ljrevtynf[, nthvbyf[
b ccskrf[. Ânjhfz xfcnm cjcnjbn bp CGI-ghjuhfvvs, ghtljcnfdkz.otq
djpvj;yjcnm gjbcrf d lfyys[, cj,hfyys[ indexer’jv. Îcyjdyst djpvj;yjcnb mnoGoSearch drk.xf.n: Ïjllth;re jcyjdys[ ghjnjrjkjd Èynthytnf (HTTP, HTTPS, FTP, NNTP) b hf,jnf c kjrfkmysvb afqkfvb;
Âcnhjtyye. gjllth;re ljrevtynjd ajhvfnf txt, html, xml, f nfr ;t
djpvj;yjcnm gjlrk.xtybz dytiyb[ ghjuhfvv-rjydthnjhjd lkz k.,s[ lheub[
nbgjd ljrevtynjd, nfrb[ rfr doc, pdf, rtf, xls, ppt, b n.l.; Ítxѐnrbq
gjbcr yf jcyjdt cbyjybvjd, gjlcnhjr, f nfr ;t utythfwbb ckjdjajhv
(gflt;b, crkjytybz, b n.l.) c bcgjkmpjdfybtv ckjdfhtq uhfvvfnbxtcrjuj
fyfkbpfnjhf ispell; 2. Ífghfdktybz hfpdbnbz cbcntvs Ïjcktlybt
dthcbb mnoGoSearch gjpdjkz.n byltrcbhjdfnm ytcrjkmrj vbkkbjyjd
ljrevtynjd yf jlyjv rjvgm.ntht, f d dthcbb 3.3 ,sk lj,fdkty vjlekm
rkfcnthbpfwbb, hfcghtltkz.obq lfyyst b ghjwtccs b[ j,hf,jnrb vt;le
ytcrjkmrbvb rjvgm.nthfvb, xnj gjpdjkbkj cjplfdfnm gjbcrjdst cbcntvs gj
rjkktrwbzv, cjcnjzobv bp ytcrjkmrb[ ltcznrjd b lf;t cjnty vbkkbjyjd
ljrevtynjd. Íf nfrb[ ,jkmib[ j,]tvf[ ljrevtynjd pflfxf hfy;bhjdfybz
dslfdftvs[ yf pfghjc ljrevtynjd zdkztncz jlyjq bp cfvs[ df;ys[, b d
yfcnjzott dhtvz ghb hfphf,jnrt mnoGoSearch bvtyyj tq eltkztncz jcj,jt
dybvfybt. Òfr, d dthcbb 3.3 ajhvfn gjbcrjdjuj byltrcf ,sk hfcibhty, xnj
lfkj djpvj;yjcnm lj,fdktybt yjds[ df;ys[ cjcnfdkz.ob[ d ajhveke
htktdfynyjcnb. 3. Ôjhvekf htktdfynyjcnb mnoGoSearch Ôjhvekf hfcxtnf htktdfynyjcnb mnoGoSearch cjcnjbn bp cktle.ob[ xfcntq (afrnjhjd):
SectionBreakdown() – aeyrwbz hfcghtltktybz ckjd gj ctrwbzv ljrevtynf.
Ýnfkjyysv cxbnftncz ljrevtyn, ult rf;ljt ckjdj bp gjbcrjdjuj pfghjcf
dcnhtxftncz d rf;ljq ctrwbb ljrevtynf (yfghbvth, d ckexft HTML
ljrevtynjd nbgjdfz yfcnhjqrf drk.xftn ctrwbb title, body, meta
keywords, b n.l., rjnjhst pflf.ncz gthtl byltrcfwbtq). Ïhb hfcxtnt
aeyrwbb hfcghtltktybz ckjd cjcnfdkztncz dtrnjh lkbyjq rjkbxtcndj_ctrwbq*rjkbxtcndj_ckjd_d_pfghjct.
Âtrnjh 'nfkjyyjuj ljrevtynf pfgjkyztncz tlbybwfvb. Âtrnjh
fyfkbpbhetvjuj ljrevtynf pfgjkyztncz yekzvb nfv, ult ckjdj yt yfqltyj d
ctrwbb b tlbybwfvb nfv, ult ckjdj yfqltyj d ctrwbb. Çfntv, dsxbckztncz
vfntvfnbxtcrfz rjhhtkzwbz vt;le ldevz dtrnjhfvb b djpdhfoftncz d
rfxtcndt pyfxtybz afrnjhf SectionBreakdown(). Òfr, yfghbvth, d ckexft
pfghjcf bp lde[ ckjd d gjbcrjdjq cbcntvt, yfcnhjtyyjq lkz hf,jns gj
nhtv ctrwbzv, hfpvths dtrnjhjd ,elen hfdys 6. Åckb j,f ckjdf pfghjcf
yfqltys njkmrj d title b ybult ,jkmit, nj d rfxtcndt htpekmnfnf
dthytncz xbckj ~0.57 – dtkbxbyf vfntvfnbxtcrjq rjhhtkzwbb vt;le
dtrnjhfvb (1,1,0,0,0,0) b (1,1,1,1,1,1).
WordDistance() – aeyrwbz ,kbpjcnb ckjd. Äjrevtyns, ult ckjdf pfghjcf
cnjzn hzljv lheu c lheujv, jwtybdf.ncz dsit, yt;tkb nt, ult ckjdf
“hfp,hjcfys” gj hfpysv xfcnzv ljrevtynf. Êhjvt jghtltktybz
ytgjchtlcndtyyjuj hfccnjzybz vt;le ckjdfvb, d hfcxtn nfr;t ,thtncz
gjhzljr ckjd b gjkyst d[j;ltybz gjbcrjds[ ahfp. MinPos() – aeyrwbz cntgtyb ,kbpjcnb gthdjuj yfqltyyjuj ckjdf r yfxfke ctrwbb ljrevtynf. WordDensity() – aeyrwbz xfcnjns bcrjvs[ ckjd d ljrevtynt. NumWords() – aeyrwbz j,otuj rjkbxtcndf yfqltyys[ ckjd. WordForm()
– aeyrwbz “vjhajkjubxtcrjuj cjjndtncndbz”. Ýnjq aeyrwbtq dsit
jwtybdf.ncz nt ljrevtyns, d rjnjhjv ckjdf dcnhtxf.ncz d njxyj nfrjq ;t
ajhvt, rfr b d pfghjct, xtv ljrevtyns c lheubvb ajhvfvb ckjd pfghjcf
(yfghbvth, lheubvb gflt;fvb ceotcndbntkmys[, dhtvtyfvb ukfujkjd,
cbyjybvfvb). Çyfxtybz dct[ gthtxbcktyys[ afrnjhjd kt;fn
d lbfgfpjyt jn 0 lj 1. Ïhb dsxbcktybb rf;ljuj afrnjhf bcgjkmpetncz
ljgjkybntkmysq yfcnhjtxysq dtrnjh wf, rjnjhsq gjpdjkztn vtyznm dtcf
hfpkbxys[ ctrwbq ljrevtynf (yfghbvth, vj;yj cltkfnm ctrwb. title ,jktt
pyfxbvjq, gj chfdytyb. c ctrwbtq body). Äkz gjkextybz tlbyjuj
xbcktyyjuj gjrfpfntkz htktdfynyjcnb pyfxtybz gthtvyj;f.ncz. Ñntgtym
dkbzybz rf;ljuj afrnjhf pflftncz yfcnhjtxysvb rj'aabwbtynfvb, f ghb
erfpfybb yektdjuj rj'aabwbtynf – cjjndtncnde.obq tve afrnjh d hfcxtnt
yt exbnsdftncz. 4. Ífcnhjqrf mnoGoSearch lkz exfcnbz d ÐÎÌÈÏ 2008 Â
2008-v ujle vs exfcndjdfkb d ljhj;rf[ “gjbcr gj web-rjkktrwbb”
(rjkktrwbb by.web b km.ru) b “gjbcr gj rjkktrwbb yjhvfnbdys[
ljrevtynjd” (rjkktrwbz legal). Ïhb yfcnhjqrt cbcntvs dj dct[ rjkktrwbz[
lkz utythfwbb ckjdjajhv ,sk bcgjkmpjdfy ckjdfhm heccrjuj zpsrf
Àktrcfylhf Ët,tltdf (bpyfxfkmyj ghtlyfpyfxtyysq lkz cbcntvs
uhfvvfnbxtcrjq ghjdthrb ispell, yj c ecgt[jv ghbvtyztvsq b d gjbcrt).
Ñbcntvf hf,jnfkf d ht;bvt “AND - yfqnb dct ckjdf”, fdnjvfnbxtcrbq
gtht[jl d ht;bv “OR - yfqnb [jnz ,s jlyj ckjdj” ghb yektdjv bkb vfkjv
rjkbxtcndt htpekmnfnjd ht;bvf “AND” yt jceotcndkzkcz. Äkz rjkktrwbq
by.web b km.ru bcgjkmpjdfkfcm yfcnhjqrf c ctrwbzvb body, title, meta
keywords b meta description. Âtc dct[ ctrwbq cxbnfkcz jlbyfrjdsv.
Êj'aabwbtyn aeyrwbb xfcnjns ckjd WordDensity ,sk ecnfyjdkty d 200 (ghb
djpvj;yjv lbfgfpjyt 1..255). Êj'aabwbtyn aeyrwbb rjkbxtcndf ckjdf
NumWord ,sk ecnfyjdkty d 1 (ghb lbfgjpjyt 0..255). Êj'aabwbtyn aeyrwbb
WordDistance ,sk ecnfyjdkty d 2500 (ghb jabwbfkmyjv lbfgfpjyt 0..255,
jlyfrj d htfkmyjcnb 'njn gfhfvtnh gjpdjkztn pflfdfnm b ,jkmibt pyfxtybz
,tp gthtgjkytybz hfphzlyjq ctnrb ghb hfcxtnf[). Êj'aabwbtyn aeyrwbb
MinPos ,sk hfdty 0 (gj evjkxfyb.), nj tcnm 'njn afrnjh yt exbnsdfkcz.
Òfr;t, ,sk bcgjkmpjdfy rj'aabwbtyn gj evjkxfyb. e aeyrwbb WordForm
(255), nj tcnm cbcntvf yt ltkfkf ghtlgjxntybz njxysv ajhvfv ckjd
pfghjcf gthtl gflt;ysvb, dhtvtyysvb ajhvfvb (b n.l.). Ñbyjybvs yt
bcgjkmpjdfkbcm. Òfrfz yfcnhjqrf zdkztncz nbgjdjq yfcnhjqrjq mnoGoSearch
lkz gjbcrf gj web-rjkktrwbb, pf bcrk.xtybtv edtkbxtyyjuj dkbzybz
aeyrwbb hfccnjzybz vt;le ckjdfvb. Óxfcnbt d gjbcrt gj rjkktrwbb
yjhvfnbdys[ ljrevtynjd – yfi gthdsq jgsn. Ìs gjgsnfkbcm ghjbpdtcnb
,jktt njyre. yfcnhjqre c extnjv jcj,tyyjcntq rjkktrwbb. Òfr, pfujkjdrb
vt;le n'ufvb <P ID=”P0000”>b </P>gjvbvj
body ,skb dsltktys b d jnltkmyst ctrwbb, nj ;t cfvjt ,skj ghjltkfyj c
pfujkjdrfvb c ID P0001-P0006. Âtcf ctrwbq, cjjndtncnde.ob[ 'nbv
pfujkjdrfv cgtwbfkmyj yt edtkbxbdfkbcm, jlyfrj afrn yf[j;ltybz ckjd rfr
d body, nfr b d jlyjv bp P000? ltkftn 'nb ckjdf ,jktt pyfxbvsvb,
gjcrjkmre edtkbxbdf.n pyfxtybt aeyrwbb hfcghtltktybz ckjd gj ctrwbzv
(SectioBreakdown). Åot cktletn jnvtnbnm, xnj d rjkktrwbb legal ,skb
gjlrk.xtys cbyjybvs, gjpdjkz.obt yf[jlbnm ytxtnrbt lfns, xnj,s,
yfghbvth, ljrevtyn c pfujkjdrjv “Çfrjy jn 1 zydfhz 2008 ujlf” ,sk
yfqlty ghb pfghjct “Çfrjy jn 01.01.2008”. Â rjkktrwbb legal ,skb
bcgjkmpjdfys rj'aabwbtyns aeyrwbq-afrnjhjd htktdfynyjcnb, fyfkjubxyst
web-rjkktrwbzv. 5. Àyfkbp htpekmnfnjd mnoGoSearch
gjrfpfk hfpyst htpekmnfns yf hfpys[ rjkktrwbz[. Íf rjkktrwbb WEB.BY yfi
htpekmnfn ,sk cnf,bkmyj yf 6-jv b 7-v vtcnf[ gj pyfxtyb. hfpkbxys[
vtnhbr (chtlb 15 exfcnybrjd), f vtnhbrf Precision pfyzkf 3-t vtcnj. Íf
rjkktrwbb KM.RU ,sk gjkexty kexibq htpekmnfn gj vtnhbrt Precision(5) b
dnjhjq htpekmnfn gj vtnhbrt Precision(10) chtlb 15-nb ghtljcnfdktyys[
htpekmnfnjd. Îlyfrj htpekmnfns gj jcnfkmysv vtnhbrfv ,skb ckf,svb b
rjkt,fkbcm vt;le 10-v b 11-v vtcnjv. Õe;t dctuj cbcntvf gjrfpfkf ct,z
yf rjkktrwbb Legal. Ýnj ,skj j;blftvsv, gjcrjkmre, dj-gthds[, vs gthdsq
hfp exfcndetv d 'njq ljhj;rt, f dj-dnjhs[, gjbcr chtlb yjhvfnbdys[
ljrevtynjd zdkztncz ytj,sxysv ghbvtytybtv mnoGoSearch. Ïj ,jkmibycnde
vtnhbr ,sk gjrfpfy 5-q htpekmnfn bp 6-nb ghtljcnfdktyys[, kbim gj jlyjq
vtnhbrt (Precision) elfkjcm gjlyznmcz lj 3-uj vtcnf. Ïj cjdjregyjcnb
htpekmnfnjd bp nht[ ljhj;tr vj;yj crfpfnm, xnj cbcntvf dscnegbkf d
wtkjv ytgkj[j. Ïjcrjkmre yf rjkktrwbb legal ,sk gjkexty [elibq
htpekmnfn, vs, ght;lt dctuj, ghjdtkb ltnfkmysq fyfkbp lkz gjbcrf ghbxby
ytelfxb bvtyyj yf 'njq rjkktrwbb. Ífif cbcntvf djj,ot yt cvjukf yfqnb
722 ljrevtynf bp 3601 gjvtxtyys[ rfr "vital" (yb chtlb 100 kexib[, yb
lf;t chtlb jcnfkmys[ htpekmnfnjd, dslfyys[ cbcntvjq). 87 ljrevtynjd
(12%) ,skb gjnthzys gj ghbxbyt f,,htdbfneh, yfghbvth, ÃÊ = ÃÐÀÆÄÀÍÑÊÈÉ
ÊÎÄÅÊÑ, ÔÇ - ÔÅÄÅÐÀËÜÍÛÉ ÇÀÊÎÍ, b n.l. Èp 'njuj vj;yj cltkfnm dsdjl,
xnj lkz ecgtiyjuj exfcnbz d cktle.ob[ ctpjyf[ yfv, ,tpeckjdyj,
gjyflj,bncz ckjdfhm f,,htdbfneh bp cjjndtncnde.otq ghtlvtnyjq j,kfcnb.
81 ljrevtyn (11%) ,sk gjnthzy d htpekmnfnt jib,rb d aeyrwbb hfcxtnf
,kbpjcnb ckjd - d ytrjnjhs[ cbnefwbz[ gjkexfkcz yektdjq htpekmnfn, b
nfrbt ljrevtyns djj,ot jn,hfcsdfkbcm rfr ythtktdfynyst. Ïhbxtv, jib,rf
d ,jkmibycndt ckexftd ghjbpjikf yf pfghjcf[ c ldevz ckjdfvb, f ghb
,jktt lkbyys[ pfghjcf[ nfrb[ c,jtd ghfrnbxtcrb yt djpybrfkj. Ñktle.ofz
ghbxbyf gjnthb - 71 ljrevtyn (9%) - eghjotyyfz htfkbpfwbz utythfnjhf
ckjdjajhv. mnoGoSearch gjlrk.xftn afqks jn cbcntvs ispell,
ghtlyfpyfxtyyjq lkz ghjdthrb jhajuhfabb. Ècgjkmpez ckjdfhb ispell,
ytkmpz gjkexfnm hfpyst xfcnb htxb. Òfr, mnoGoSearch yt yfitk ljrevtyns
c ghbkfufntkmysv "Ëtybyuhflcrbq" ghb pfghjct "Ëtybyuhfl". Ýnj yt
zdkztncz ghj,ktvjq cfvjuj ispell, gjcrjkmre ghb ghjdthrt jhajuhfabb
gtht[jlf vt;le xfcnzvb htxb yt nht,etncz, yj, rfr gjrfpfkb htpekmnfns -
'nj df;yj lkz gjbcrf. Ñltkfy dsdjl j ytj,[jlbvjcnb gjlrk.xtybz ,jktt
ckj;ys[ cbcntv lkz utythfwbb ckjdjajhv. 61 ljrevtyn (8%) yt ,sk yfqlty
d htpekmnfnt, rfr jrfpfkjcm, ytghfdbkmyjuj bcgjkmpjdfybz afqkjd jn
ispell. Ïjckt rjycekmnfwbb c fdnjhfvb ispell ghj,ktve elfkjcm
ecnhfybnm. Çfrk.xtybt. Àyfkbp
htpekmnfnjd exfcnbz d ÐÎÌÈÏ-2008 gjpdjkbk edbltnm rfr ljcnjbycndf, nfr
b ytljcnfnrb yfitq gjbcrjdjq cbcntvs, xnj ytjwtybvj lkz ghfdbkmyjuj
ds,jhf yfghfdktybq lfkmytqitq hf,jns. Ïj'njve cxbnftv, xnj exfcnbt d
rjyathtywbb jrfpfkjcm lkz yfc gkjljndjhysv. Õjnbv dshfpbnm
,kfujlfhyjcnm jhurjvbntne pf ghtljcnfdktyye. djpvj;yjcnm exfcnbz d
rjyathtywbb ÐÎÌÈÏ-2008, f nfr;t pf ,scnhe. gjvjom ghb djpybryjdtybb
ntreob[ djghjcjd b pfnhelytybq. Â xfcnyjcnb, [jnbv gj,kfujlfhbnm Èujhz Ítrhtcnmzyjdf b Ìfhbye Ítrhtcnmzyjde. Search engine “mnoGoSearch” Barkov
A.I. This article presents a report on experiments in full text
retrieval made as a part of ROMIP’2008. The main goal of these
experiments was to approbate methods of document ranking implemented in
mnoGoSearch throughout the last years. |
