Intuitive Python Regexes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Notebook is available at my github _tfer_\n",
    "* Much of this is taken from official website, which is under the MIT license, so this is too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Some code make sure we can use the srl mosule\n",
    "srl_module_present=True\n",
    "try:\n",
    "   from srl import SRL\n",
    "except ImportError:\n",
    "    srl_module_present=False\n",
    "    import sys\n",
    "    !{sys.executable} -m pip install srl\n",
    "    from srl import SRL\n",
    "\n",
    "if srl_module_present:\n",
    "    print(\"The srl module was already present.\")\n",
    "else:\n",
    "    print(\"The srl module had to be installed first.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__===========================================================__\n",
    "\n",
    "# Simple Regex Language"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## There are a few basic syntax rules you should know before diving in\n",
    "\n",
    "* SRL is case insensitive. Thus, LITERALLY \"test\" is exactly the same as literally \"test\". But please beware, that everything inside a string is in fact case sensitive. LITERALLY \"TEST\" does NOT equal literally \"test\".\n",
    "* The comma separating statements is completely optional and has no effect whatsoever. In fact, it gets removed before interpreting. But since it helps the human eye to distinct between different statements, it's allowed.\n",
    "* Strings are interpreted as literal characters and will have any escapes interpreted. They can either be defined using 'single', \"double\", (plus \"\"\"triple\"\"\" in Python) quotation marks. Escaping them (quotes?-trf) using a backslash is possible as well.\n",
    "* Parentheses should only be used when building a sub-query, for example while using a capture or non-capture group to, for example, apply a quantifier to multiple items. [^elsewhere the're used for sub-query and/or capture]\n",
    "* Comments are currently not supported and may be implemented in the future.\n",
    "__===========================================================__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Frame [^ Called 'Character' in the official Doc]\n",
    "\n",
    "A frame is anything that can match a tile in our model.  An SRL statement to describe one, or a sequence of frames takes the following form:\n",
    "\n",
    "```srl-BNF\n",
    "<character-set-name> [specification] [quantifier] [anchor]\n",
    "```\n",
    "\n",
    "As you can see, the `<character-set-name>` almost always come first, (though there can be another `[anchor]` before it). They start a new statement, and everything that follows defines or refines the frame(s) it introduces.  Some `<character-set-name>`'s allow a specification.  For example `LETTER`, allows you to specify a span of allowed letters, e.g.: `from a to f`.\n",
    "\n",
    "Every frame or frame sequence can be quantified. You may want to match exactly four letters from a to f. This would match abcd, but not abcg. You can do that by supplying `exactly 4 times` as a quantifier:\n",
    "\n",
    "```srl\n",
    "letter from a to f exactly 4 times\n",
    "```\n",
    "\n",
    "Note: this adds 4 frames to our regex, four copies of `letter from a to f`\n",
    "\n",
    "\n",
    "Okay, let's dive into the different `<character-set-name>'s`. Below, we'll go though all the available `<character-set-name>'s` and give some example queries.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LITERALLY\n",
    "```srl\n",
    "literally \"string\"\n",
    "```\n",
    "\n",
    "The `literally` <character-set-name> allows you build a sequence of frames up with one statement.  It passes a string to the query that will be interpreted exactly as what you've requested. Nothing else will match besides your string. Any special character will automatically be escaped.\n",
    "\n",
    "#### Example query:\n",
    "```srl\n",
    "literally \"sample\"\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('literally \"sample\"')\n",
    "# the python regex the next line returns follows a Perl 5 extension\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ONE OF\n",
    "\n",
    "example format:\n",
    "```srl-BNF\n",
    "one of \"characters\"\n",
    "```\n",
    "\n",
    "So `literally`, (above), comes in handy if the string is known. But if there is a unknown string which may only contain certain characters, using `ONE OF` makes much more sense. This will match one of the supplied characters.\n",
    "\n",
    "#### Example query:\n",
    "```srl\n",
    "one of \"a%1\"\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[a\\\\%1]'"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srl = SRL('one of \"a%1\"')\n",
    "# the python regex the next line returns a character class that needs\n",
    "#  to the double backslash the percent sign to allow it to be escaped\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "#'[a\\\\%1]' == r'[a%1]'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LETTER _and_ UPPERCASE LETTER\n",
    "\n",
    "format:\n",
    "\n",
    "```srl-BNF\n",
    "letter [from a to z]\n",
    "```\n",
    "\n",
    "This will help you to match a letter between a specific span, if the specific character expected isn't known. If you know you're expecting an letter, then go for it. If not supplying anything, a normal letter between a and z will be matched. Of course, you can define a span, using the `from <x> to <y>` syntax.\n",
    "\n",
    "Please note, that this will only match one letter. If you expect more than one letter, use a quantifier.\n",
    "\n",
    ">Note: LETTER would be called an alphabetic character in computer science class.\n",
    "\n",
    "#### Example queries:\n",
    "\n",
    "```srl\n",
    "letter from a to f\n",
    "uppercase letter\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('letter from a to f')\n",
    "# below returns a character class using python 'span' notation\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('uppercase letter')\n",
    "# below returns a character class using python 'span' notation\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or use in this format like the next example does:\n",
    "\n",
    "```srl-BNF\n",
    "uppercase letter [from A to Z]\n",
    "```\n",
    "\n",
    "This of course behaves just like the normal letter, with the only difference, that uppercase letter only matches letters that are written in uppercase. Of course, if the case insensitive flag is applied to the query, these two act completely the same.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```srl\n",
    "uppercase letter from A to F\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('uppercase letter from A to F')\n",
    "# below returns a character class using python 'span' notation\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ANY CHARACTER\n",
    "\n",
    "format:\n",
    "\n",
    "```srl-BNF\n",
    "any character\n",
    "```\n",
    "\n",
    "Just like a letter, any character matches anything between A to Z, plus 0 to 9 and _, -case insensitive. This way you can validate if someone for example entered a valid username.\n",
    "\n",
    ">In many computer languages, including Python, these are the characters from which you can form valid identifers.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```srl\n",
    "starts with any character once or more, must end\n",
    "```\n",
    "\n",
    ">Note: this example shows an `anchor` in front, i.e. the `starts with`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('starts with any character once or more, must end')\n",
    "# below returns string that when python-interpreted \n",
    "#  === r'^\\w+$'\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NO CHARACTER\n",
    "\n",
    "```srl-BNF\n",
    "no character\n",
    "```\n",
    "\n",
    "The inverse to the `any character` is `no character`. This will match everything except a to z, A to Z, 0 to 9 and _.\n",
    "\n",
    "Example query:\n",
    "\n",
    "```srl\n",
    "starts with no character once or more, must end\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "regex_in_SRL = \"\"\"\n",
    "starts with\n",
    "    no character once or more\n",
    "must end\n",
    "\"\"\"\n",
    "\n",
    "srl = SRL(regex_in_SRL)\n",
    "# below will return a string that when interpretived\n",
    "#  is === r'^\\W+$'\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DIGIT or NUMBER\n",
    "\n",
    "format:\n",
    "\n",
    "```srl-BNF\n",
    "digit [from 0 to 9]\n",
    "```\n",
    "\n",
    "When expecting a digit, but not a specific one, this comes in handy. Each `digit` matches only one digit, meaning it will only match a single `digit from 0 to 9`, but you can repeat that by using a quantifier.  Obviously, limiting the allowed values for `digit` isn't a problem either.  So if you're searching for a `number from 5 to 7`, go for it!\n",
    "\n",
    ">Note: `number` is an alias for `digit`.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```srl\n",
    "starts with digit from 5 to 7 exactly 2 times, must end\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('starts with digit from 5 to 7 exactly 2 times, must end')\n",
    "# \n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output above has this form:\n",
    "\n",
    "\n",
    ">`anchor`, a `character-class` in span-format, a `curlied-exact-count`, an `anchor`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ANYTHING\n",
    "\n",
    "format:\n",
    "\n",
    "```srl-BNF\n",
    "anything\n",
    "```\n",
    "\n",
    "Any character whatsoever. Well.. except for line breaks. This will match any character, except new lines. And, of course, only once. So don't forget to apply a quantifier, if necessary.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```srl\n",
    "anything\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('anything')\n",
    "#\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NEW LINE\n",
    "\n",
    "format:\n",
    "\n",
    "```srl-BNF\n",
    "new line\n",
    "```\n",
    "\n",
    "Match a new line. Forgive us, if we can't provide an example for that one, but you can check it out yourself below.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### WHITESPACE and NO WHITESPACE\n",
    "\n",
    "```srl_BNF \n",
    "[no] whitespace\n",
    "```\n",
    "\n",
    "This matches any whitespace character. This includes a space, tab or new line. If using no whitespace everything except a whitespace character will match.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```srl\n",
    "whitespace\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('whitespace')\n",
    "# below's result === r'\\s'\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r'\\s' == '\\\\s'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TAB\n",
    "\n",
    "```srl-BNF\n",
    "tab\n",
    "```\n",
    "\n",
    "If you want to match tabs, but no other whitespace characters, this might be for you. It will only match the tab character, and nothing else.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```snl_BNF\n",
    "tab\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('tab')\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'\\\\t' == r'\\t'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### backslash\n",
    "\n",
    "```srl\n",
    "backslash\n",
    "```\n",
    "\n",
    "Matching a backslash with `literally` would work, but requires escaping, since the backslash is the escaping character. Thus, you'd have to use literally \"\\\\\" to match one backslash. Or you could just write backslash.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "```srl\n",
    "backslash\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#srl = SRL('backslash')\n",
    "#srl.pattern\n",
    "\n",
    "# seems broken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# should give this\n",
    "'\\\\' == r'\\'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RAW\n",
    "\n",
    "format:\n",
    "\n",
    "```srl\n",
    "raw \"expression\"\n",
    "```\n",
    "\n",
    "Sometimes, you may want to enforce a specific part of a regular expression. You can do this by using raw. This will append the given string without escaping it.\n",
    "\n",
    "#### Example query:\n",
    "\n",
    "literally \"an\", whitespace, raw \"[a-zA-Z]\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "srl = SRL('literally \"an\", whitespace, raw \"[a-zA-Z]\"')\n",
    "srl.pattern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__===========================================================__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quantifiers\n",
    "\n",
    "Quantifiers are probably one of the most important things here. If you've specified a character or a group in your query and now want to multiply it, you don't have to copy and paste all of it. Just tell them how many copies to allow.\n",
    "\n",
    "Oh, and don't be confused. Sometimes, you may find that these quantifiers don't match with the tinkered example. That's okay, since we're not forcing the string to start or end. Thus, even if only parts of that string are matching, the expression will be valid.\n",
    "\n",
    ">Remember: You can execute every Python Cell in this notebook by clicking it, and then pressing shift-enter !\n",
    "\n",
    "___\n",
    "\n",
    "### EXACTLY _and_ ONCE _and_ TWICE\n",
    "\n",
    "__format:__\n",
    "\n",
    "```srl-BNF\n",
    "exactly <x> times\n",
    "```\n",
    "\n",
    "You're sure. You don't guess, you dictate `exactly 4 times`. Not more, not less. The statement before has to match _exactly x times_.\n",
    "\n",
    ">Note: since exactly x times is pretty much to write, two common shortcut terms exist.  Instead of `exactly 1 time`, you can write `once`, and for `exactly 2 times`, write `twice`\n",
    "\n",
    "__format:__\n",
    "\n",
    "```srl-BNF\n",
    "digit exactly 3 times, letter twice\n",
    "```\n",
    "\n",
    "### BETWEEM <x\\> AND <y\\> TIMES\n",
    "\n",
    "__format:__\n",
    "\n",
    "```srl-BNF\n",
    "between <x> and <y> times\n",
    "```\n",
    "For a specific number of repetitions between a span of <x\\> to <y\\>, you may use this quantifier. It will make sure the previous character exists between x and y times.\n",
    "\n",
    ">Note: since between x and y times is pretty much to write, you can get rid of the times: `between 1 and 5`\n",
    "\n",
    "__Example query:__\n",
    "\n",
    "```srl\n",
    "Example query:\n",
    "starts with digit between 3 and 5 times, letter twice\n",
    "optional\n",
    "optional\n",
    "```\n",
    "\n",
    "### OPTIONAL\n",
    "\n",
    "You can't always be sure that something exists. Sometimes it's okay if something is missing. In that case, the `optional` quantifier comes in handy. It will match the sub-query, if it's there, and ignore it, if it's missing.\n",
    "\n",
    "__Example query:__\n",
    "\n",
    "```srl\n",
    "digit optional, letter twice\n",
    "```\n",
    "\n",
    "### ONCE/NEVER OR MORE\n",
    "\n",
    "__format:__\n",
    "\n",
    "```srl-BNF\n",
    "once or more\n",
    "never or more\n",
    "```\n",
    "\n",
    "If something has to exist at least once, or never, but if it does, then it may exist multiple times, the quantifiers once or more and never or more will do the job.\n",
    "\n",
    "__Example query:__\n",
    "\n",
    "```srl\n",
    "starts with letter once or more, must end\n",
    "```\n",
    "\n",
    "### AT LEAST X TIMES\n",
    "\n",
    "__format:__\n",
    "\n",
    "```BNF\n",
    "at least <x> times\n",
    "```\n",
    "\n",
    "Something may exist in an infinite length, but must exist at least x times.\n",
    "\n",
    "__Example query:__\n",
    "\n",
    "```srl\n",
    "letter at least 10 times\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}