Python - NLTK - 言語処理とPython(リストの結合、集合、ソート処理、string module、punctuation)

入門 自然言語処理
原著

開発環境

• macOS High Sierra - Apple
• Emacs (Text Editor)
• Python 3.6 (プログラミング言語)

入門 自然言語処理 (Steven Bird (著)、Ewan Klein (著)、Edward Loper (著)、萩原 正人 (翻訳)、中山 敬広 (翻訳)、水野 貴明 (翻訳)、オライリージャパン)の1章(言語処理とPython)、1.8(演習問題)18.を取り組んでみる。

コード(Emacs)

Python 3

#!/usr/bin/env python3

print('18.')
from nltk.book import *
import string

s = sent1 + sent2 + sent3 + sent4 + sent5 + sent6 + sent7 + sent8
print(s)

s = set(s)
print()

s = sorted(s)
print(s)
s = [w for w in s if w not in string.punctuation]
for w in s:
    print(w)

入出力結果(Terminal, Jupyter(IPython))

$ ./sample12.py
18.
*** Introductory Examples for the NLTK Book ***
Loading text1, ..., text9 and sent1, ..., sent9
Type the name of the text or sentence to view it.
Type: 'texts()' or 'sents()' to list the materials.
text1: Moby Dick by Herman Melville 1851
text2: Sense and Sensibility by Jane Austen 1811
text3: The Book of Genesis
text4: Inaugural Address Corpus
text5: Chat Corpus
text6: Monty Python and the Holy Grail
text7: Wall Street Journal
text8: Personals Corpus
text9: The Man Who Was Thursday by G . K . Chesterton 1908
['Call', 'me', 'Ishmael', '.', 'The', 'family', 'of', 'Dashwood', 'had', 'long', 'been', 'settled', 'in', 'Sussex', '.', 'In', 'the', 'beginning', 'God', 'created', 'the', 'heaven', 'and', 'the', 'earth', '.', 'Fellow', '-', 'Citizens', 'of', 'the', 'Senate', 'and', 'of', 'the', 'House', 'of', 'Representatives', ':', 'I', 'have', 'a', 'problem', 'with', 'people', 'PMing', 'me', 'to', 'lol', 'JOIN', 'SCENE', '1', ':', '[', 'wind', ']', '[', 'clop', 'clop', 'clop', ']', 'KING', 'ARTHUR', ':', 'Whoa', 'there', '!', 'Pierre', 'Vinken', ',', '61', 'years', 'old', ',', 'will', 'join', 'the', 'board', 'as', 'a', 'nonexecutive', 'director', 'Nov.', '29', '.', '25', 'SEXY', 'MALE', ',', 'seeks', 'attrac', 'older', 'single', 'lady', ',', 'for', 'discreet', 'encounters', '.']

['!', ',', '-', '.', '1', '25', '29', '61', ':', 'ARTHUR', 'Call', 'Citizens', 'Dashwood', 'Fellow', 'God', 'House', 'I', 'In', 'Ishmael', 'JOIN', 'KING', 'MALE', 'Nov.', 'PMing', 'Pierre', 'Representatives', 'SCENE', 'SEXY', 'Senate', 'Sussex', 'The', 'Vinken', 'Whoa', '[', ']', 'a', 'and', 'as', 'attrac', 'been', 'beginning', 'board', 'clop', 'created', 'director', 'discreet', 'earth', 'encounters', 'family', 'for', 'had', 'have', 'heaven', 'in', 'join', 'lady', 'lol', 'long', 'me', 'nonexecutive', 'of', 'old', 'older', 'people', 'problem', 'seeks', 'settled', 'single', 'the', 'there', 'to', 'will', 'wind', 'with', 'years']
1
25
29
61
ARTHUR
Call
Citizens
Dashwood
Fellow
God
House
I
In
Ishmael
JOIN
KING
MALE
Nov.
PMing
Pierre
Representatives
SCENE
SEXY
Senate
Sussex
The
Vinken
Whoa
a
and
as
attrac
been
beginning
board
clop
created
director
discreet
earth
encounters
family
for
had
have
heaven
in
join
lady
lol
long
me
nonexecutive
of
old
older
people
problem
seeks
settled
single
the
there
to
will
wind
with
years
$