Skip to content

Commit

Permalink
[fix] fix tn, week range (#238)
Browse files Browse the repository at this point in the history
* [fix] fix tn, week range

* [fix] fix tn, week range
  • Loading branch information
xingchensong authored Jun 6, 2024
1 parent 0f386d8 commit 80d25fc
Show file tree
Hide file tree
Showing 4 changed files with 27 additions and 2 deletions.
18 changes: 18 additions & 0 deletions tn/english/data/date/week.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
Mon Monday
Mon. Monday
Tu Tuesday
Tu. Tuesday
Wed Wednesday
Wed. Wednesday
Th Thursday
Th. Thursday
Thur Thursday
Thur. Thursday
Thurs Thursday
Thurs. Thursday
Fri Friday
Fri. Friday
Sat Saturday
Sat. Saturday
Sun Sunday
Sun. Sunday
1 change: 1 addition & 0 deletions tn/english/data/whitelist/alternatives.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,5 @@ NO. number
NO. number
No. number
VOL. Volume
Vol. Volume
TV Television
8 changes: 7 additions & 1 deletion tn/english/rules/range.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from pynini.lib import pynutil

from tn.processor import Processor
from tn.utils import get_abs_path
from tn.english.rules.cardinal import Cardinal
from tn.english.rules.time import Time
from tn.english.rules.date import Date
Expand Down Expand Up @@ -45,14 +46,19 @@ def build_tagger(self):
time = time.tagger @ time.verbalizer
date = Date(deterministic=self.deterministic)
date = date.tagger @ date.verbalizer
week = pynini.string_file(get_abs_path("english/data/date/week.tsv"))
delete_space = pynini.closure(pynutil.delete(" "), 0, 1)

approx = pynini.cross("~", "approximately")

# WEEK
week_graph = week + delete_space + (pynini.cross("-", " to ")
| approx) + delete_space + week

# TIME
time_graph = time + delete_space + pynini.cross(
"-", " to ") + delete_space + time
self.graph = time_graph | (approx + time)
self.graph = time_graph | (approx + time) | week_graph

# YEAR
date_year_four_digit = (self.DIGIT**4 +
Expand Down
2 changes: 1 addition & 1 deletion tn/english/test/data/normalizer.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
this is 12th game, number 256, 2024-05-06, 2021-03-07 31.990 billion. ¾ people like chattts, let's eat at 03:43 p.m. run 10 km, give me $12.345 please, call 123-123-5678-1 Mt Hill "HAHAHA" billion 4 March => this is twelfth game, number two hundred and fifty six, the sixth of may twenty twenty four, the seventh of march twenty twenty one thirty one point nine nine oh billion. three quarters people like chattts, let' s eat at three forty three PM run ten kilometers, give me twelve point three four five dollars please, call one two three, one two three, five six seven eight, one Mt Hill" HAHAHA" billion the fourth of march
The National Map, accessed April 1, 2011" Site Description of Koppers Co. From the quartet's recording" Jefferson Friedman: Quartets,"" String Quartet no, Riots again broke out, Atassi resigned, and Syrian independence was deferred until after World War II. 1988 (1988) ( 1988) ( 1988). Starling, Arthur E.( 1988 ). this is 12th game, number 256, 2024-05-06, 2021-03-07 31.990 billion. 3/4 people like chattts Retrieved December 2011. Information on Album" Thepodule.com"" Biography by Amy Hanson". => The National Map, accessed the first of april , twenty eleven" Site Description of Koppers company From the quartet' s recording" Jefferson Friedman: Quartets,"" String Quartet no, Riots again broke out, Atassi resigned, and Syrian independence was deferred until after World War two nineteen eighty eight( nineteen eighty eight )( nineteen eighty eight )( nineteen eighty eight). Starling, Arthur E.( nineteen eighty eight). this is twelfth game, number two fifty six, the sixth of may twenty twenty four, the seventh of march twenty twenty one thirty one point nine nine oh billion. three quarters people like chattts Retrieved december twenty eleven. Information on Album" Thepodule dot com"" Biography by Amy Hanson".
.345" and ".456" "9.456" or 6.7890" => point three four five" and". four hundred and fifty six" " nine point four five six" or six point seven eight nine oh"
The museum is open Mon.-Sun. children of 3-4 years 123 The plan will help you lose 3-4 pounds the first week, and 1-2 pounds the weeks thereafter. => The museum is open Monday.- Sunday. children of three to four years one hundred and twenty three The plan will help you lose three to four pounds the first week, and one to two pounds the weeks thereafter.
The museum is open Mon.-Sun. children of 3-4 years 123 The plan will help you lose 3-4 pounds the first week, and 1-2 pounds the weeks thereafter. => The museum is open Monday to Sunday children of three to four years one hundred and twenty three The plan will help you lose three to four pounds the first week, and one to two pounds the weeks thereafter.

0 comments on commit 80d25fc

Please sign in to comment.