analyzer

How to use a Lucene Analyzer to tokenize a String?

by

Based off of the answer above, this is slightly modified to work with Lucene 4.0. public final class LuceneUtil { private LuceneUtil() {} public static List<String> tokenizeString(Analyzer analyzer, String string) { List<String> result = new ArrayList<String>(); try { TokenStream stream = analyzer.tokenStream(null, new StringReader(string)); stream.reset(); while (stream.incrementToken()) { result.add(stream.getAttribute(CharTermAttribute.class).toString()); } } catch (IOException e) { … Read more

How do I disable all Roslyn Code Analyzers?

by

You can disable analyzers on a per-project basis. To do it, right click on Project>References>Analyzers in the Solution Explorer and hit Open Active Rule Set You can disable individual analyzers or entire bundles of analyzers. This creates a <ProjectName>.ruleset file and modifies the <ProjectName>.csproj, which means that you will share this configuration with your team … Read more

How can I make my code diagnostic syntax node action work on closed files?

by

For the closed file issues, it’s our intent that all diagnostics will be reported, from either open or closed files. There is a user option for it in the preview at Tools\Options\Text Editor\C#\Advanced that you can toggle to include diagnostics in closed files. We hope to make this the default before VS 2015 is released. … Read more

Comparison of Lucene Analyzers

by

In general, any analyzer in Lucene is tokenizer + stemmer + stop-words filter. Tokenizer splits your text into chunks, and since different analyzers may use different tokenizers, you can get different output token streams, i.e. sequences of chunks of text. For example, KeywordAnalyzer you mentioned doesn’t split the text at all and takes all the … Read more