which_are: Detect/Locate Potential Non-Normalized Text.sub_holder: Hold the Place of Characters Prior to Subbing.replace_word_elongation: Replace Word Elongations.replace_white: Remove Escaped Characters.replace_to: Grab Begin/End of String to/from Character.replace_time: Replace Time Stamps With Words.replace_symbol: Replace Symbols With Word Equivalents.replace_rating: Replace Ratings With Words.replace_ordinal: Replace Mixed Ordinal Numbers With Text Representation.replace_number: Replace Numbers With Text Representation.replace_non_ascii: Replace Common Non-ASCII Characters.replace_names: Replace First/Last Names.replace_money: Replace Money With Words.replace_kern: Replace Kerned (Spaced) with No Space Version.replace_internet_slang: Replace Internet Slang.replace_incomplete: Denote Incomplete End Marks With "|".replace_grade: Replace Grades With Words.replace_emoticon: Replace Emoticons With Words.replace_emoji: Replace Emojis With Words/Identifier.replace_contraction: Replace Contractions.reexports: Objects exported from other packages.print.which_are_locs: Prints a which_are_locs Object.print.sub_holder: Prints a sub_holder object.print.check_text: Prints a check_text Object.match_tokens: Find Tokens that Match a Regex.make_plural: Make Plural (or Verb to Singular) Versions of Words.has_endmark: Test for Incomplete Sentences.filter_row: Remove Rows That Contain Markers.filter_element: Remove Elements in a Vetor.fgsub: Replace a Regex with an Functional Operation on the Regex.drop_row: Filter Rows That Contain Markers.drop_element: Filter Elements in a Vetor.check_text: Check Text For Potential Problems.add_missing_endmark: Add Missing Endmarks.add_comma_space: Ensure Space After Comma.Also, I was curious to time two different implementations of maketrans for Python 3 table = str. Perhaps you are using distributed computing and can't have regex object shared between your workers and need to have re.compile step at each worker. My thought here was to time every single step needed to make the function work. Just as an update, I rewrote the example in Python 3 and made changes to it to move regex compile step inside of the function. This gives the following results: sets : 19.8566138744 ![]() Print "replace :",timeit.Timer('f(s)', 'from _main_ import s,test_repl as f').timeit(1000000) Print "translate :",timeit.Timer('f(s)', 'from _main_ import s,test_trans as f').timeit(1000000) Return s.translate(table, string.punctuation)ĭef test_repl(s): # From S.Lott's solution Return ''.join(ch for ch in s if ch not in exclude)ĭef test_re(s): # From Vinko's solution, with fix. Regex = re.compile('' % re.escape(string.punctuation)) For this type of problem, doing it at as low a level as possible pays off. This is faster than s.replace with each char, but won't perform as well as non-pure python approaches such as regexes or anslate, as you can see from the below timings. S = ''.join(ch for ch in s if ch not in exclude) If speed isn't a worry, another option though is: exclude = set(string.punctuation) It's performing raw string operations in C with a lookup table - there's not much that will beat that but writing your own C code. From an efficiency perspective, you're not going to beat s.translate(None, string.punctuation)įor higher versions of Python use the following code: s.translate(str.maketrans('', '', string.punctuation))
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |