python - nltk.DecisionTreeClassifier.train doesn't work -
i attempting run following code provided example in following book chapter: http://www.nltk.org/book/ch06.html [see 1.4 part-of-speech tagging]
import nltk nltk.corpus import brown # suffix_fdist = nltk.freqdist() word in brown.words(): word = word.lower() suffix_fdist[word[-1:]] += 1 suffix_fdist[word[-2:]] += 1 suffix_fdist[word[-3:]] += 1 common_suffixes = [suffix (suffix,count) in suffix_fdist.most_common(100)] print(common_suffixes) def pos_features(word): features={} suffix in common_suffixes: features['endswith({})'.format(suffix)] = word.lower().endswith(suffix) return features tagged_words = brown.tagged_words(categories='news') featuresets = [(pos_features(n), g) (n,g) in tagged_words] size = int(len(featuresets)*0.1) train_set, test_set = featuresets[size:], featuresets[:size] classifier = nltk.decisiontreeclassifier.train(train_set) nltk.classify.accuracy(classifier, test_set) when run code idle seems go infinite loop. problem appears occur within nltk.decisiontreeclassifier.trai method call. idea why happening , how might go fixing highly appreciated.
mike
Comments
Post a Comment