2017.11.17) Konlpy를 활용한 야민정음 해석기 part.1

필자는 2017 15인치 맥북 터치바, 

(

i7 - 7700hq

radeon pro 555

LPDDR3 16gb

APPLE SSD 256gb

)

파이참, 

파이썬 3.6 환경에서 구동하였다.

konlpy 공식 사이트의 코드를 활용한 문장 분석

import konlpy
import nltk
 
a = u'아버지가 방에 들어가신다'
b = u'며용'
 
kkma_words_1 = konlpy.tag.Kkma().pos(a)
kkma_words_2 = konlpy.tag.Kkma().pos(b)
 
Twitter_words_1 = konlpy.tag.Twitter().pos(a)
Twitter_words_2 = konlpy.tag.Twitter().pos(b)
 
grammar = """
NP: {<N.*>*<Suffix>?}   
VP: {<V.*>*}            
AP: {<A.*>*}           
"""
parser = nltk.RegexpParser(grammar)
 
print("kkma 비교")
print(parser.parse(kkma_words_1).pprint())
print(parser.parse(kkma_words_2).pprint())
 
print("\nTwitter 비교")
print(parser.parse(Twitter_words_1).pprint())
print(parser.parse(Twitter_words_2).pprint())
 

문장 자동 생성

import bisect
import itertools
import random
 
import nltk
from konlpy.corpus import kolaw
from konlpy.tag import Mecab
 
 
def generate_sentence(cfdist, word, num=15):
    sentence = []
 
    while word!='.':
        sentence.append(word)
 
        choices, weights = zip(*cfdist[word].items())
        cumdist = list(itertools.accumulate(weights))
        x = random.random() * cumdist[-1]
        word = choices[bisect.bisect(cumdist, x)]
 
    return ' '.join(sentence)
 
 
def calc_cfd(doc):
    words = [w for w, t in Mecab().pos(doc)]
    bigrams = nltk.bigrams(words)
    return nltk.ConditionalFreqDist(bigrams)
 
 
if __name__=='__main__':
    nsents = 5
    initstr = u'정치'
 
    doc = kolaw.open('constitution.txt').read()
    cfd = calc_cfd(doc)
 
    for i in range(nsents):
        print('%d. %s' % (i, generate_sentence(cfd, initstr)))