elasticsearch同义词()

1、同义词词库准备一个同义词文件,每行一个同义词词组,例子sku_synonym.txt:西红柿,番茄,tomato =>西红柿,番茄马铃薯,土豆 =>马铃薯,土豆社保,公积金 =>社保,公积金

注意:编码 UTF-8 (很关键!!!我就是被坑在这里)

2、上传同义词文件上传同义词文件,点击此文档创建索引时需要配置settings和mapping,并且需要在settings中配置”synonyms_path”: “analysis/your_dict_name.txt”。词库位置:analysis/sku_synonym.txt

3、配置分析器

delete sku_syno

PUT sku_syno
{
  "settings": {
    "analysis": {
      "filter": {
        "my_synonym_filter": {
          "type": "synonym",
          "synonyms_path" : "analysis/sku_synonym.txt"
        }
      },
      "analyzer": {
        "my_synonyms": {
          "tokenizer": "ik_smart",
          "filter": [
            "lowercase",
            "my_synonym_filter"
          ]
        }
      }
    }
  }
}

4、分词效果测试

GET /sku_syno/_analyze
{
  "text":"我爱吃土豆社保番茄",
  "analyzer": "my_synonyms"
}

演示效果:

{
  "tokens" : [
    {
      "token" : "我",
      "start_offset" : 0,
      "end_offset" : 1,
      "type" : "CN_CHAR",
      "position" : 0
    },
    {
      "token" : "爱吃",
      "start_offset" : 1,
      "end_offset" : 3,
      "type" : "CN_WORD",
      "position" : 1
    },
    {
      "token" : "马铃薯",
      "start_offset" : 3,
      "end_offset" : 5,
      "type" : "SYNONYM",
      "position" : 2
    },
    {
      "token" : "土豆",
      "start_offset" : 3,
      "end_offset" : 5,
      "type" : "SYNONYM",
      "position" : 2
    },
    {
      "token" : "社保",
      "start_offset" : 5,
      "end_offset" : 7,
      "type" : "SYNONYM",
      "position" : 3
    },
    {
      "token" : "公积金",
      "start_offset" : 5,
      "end_offset" : 7,
      "type" : "SYNONYM",
      "position" : 3
    },
    {
      "token" : "西红柿",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "SYNONYM",
      "position" : 4
    },
    {
      "token" : "番茄",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "SYNONYM",
      "position" : 4
    }
  ]
}
————————

1、同义词词库准备一个同义词文件,每行一个同义词词组,例子sku_synonym.txt:西红柿,番茄,tomato =>西红柿,番茄马铃薯,土豆 =>马铃薯,土豆社保,公积金 =>社保,公积金

注意:编码 UTF-8 (很关键!!!我就是被坑在这里)

2、上传同义词文件上传同义词文件,点击此文档创建索引时需要配置settings和mapping,并且需要在settings中配置”synonyms_path”: “analysis/your_dict_name.txt”。词库位置:analysis/sku_synonym.txt

3、配置分析器

delete sku_syno

PUT sku_syno
{
  "settings": {
    "analysis": {
      "filter": {
        "my_synonym_filter": {
          "type": "synonym",
          "synonyms_path" : "analysis/sku_synonym.txt"
        }
      },
      "analyzer": {
        "my_synonyms": {
          "tokenizer": "ik_smart",
          "filter": [
            "lowercase",
            "my_synonym_filter"
          ]
        }
      }
    }
  }
}

4、分词效果测试

GET /sku_syno/_analyze
{
  "text":"我爱吃土豆社保番茄",
  "analyzer": "my_synonyms"
}

演示效果:

{
  "tokens" : [
    {
      "token" : "我",
      "start_offset" : 0,
      "end_offset" : 1,
      "type" : "CN_CHAR",
      "position" : 0
    },
    {
      "token" : "爱吃",
      "start_offset" : 1,
      "end_offset" : 3,
      "type" : "CN_WORD",
      "position" : 1
    },
    {
      "token" : "马铃薯",
      "start_offset" : 3,
      "end_offset" : 5,
      "type" : "SYNONYM",
      "position" : 2
    },
    {
      "token" : "土豆",
      "start_offset" : 3,
      "end_offset" : 5,
      "type" : "SYNONYM",
      "position" : 2
    },
    {
      "token" : "社保",
      "start_offset" : 5,
      "end_offset" : 7,
      "type" : "SYNONYM",
      "position" : 3
    },
    {
      "token" : "公积金",
      "start_offset" : 5,
      "end_offset" : 7,
      "type" : "SYNONYM",
      "position" : 3
    },
    {
      "token" : "西红柿",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "SYNONYM",
      "position" : 4
    },
    {
      "token" : "番茄",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "SYNONYM",
      "position" : 4
    }
  ]
}