{
 "cells": [
  {
   "cell_type": "raw",
   "id": "9819c0c8-09a6-41d2-8f3b-dc4d7e2a2a8c",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "raw_mimetype": "text/restructuredtext",
    "tags": []
   },
   "source": [
    ".. index:: Low-level"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43c6eb31",
   "metadata": {},
   "source": [
    "# Low-level API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b11c2ff6-34f1-4079-b056-c482b85d3c71",
   "metadata": {},
   "source": [
    "The most common usage of the **pyodc** module will be for encoding and decoding ODB-2 data. However, the module also contains a number of classes that facilitate the exploration of the data. Together, they provide an API for exploring the structure and contents of ODB-2 data without actually decoding it."
   ]
  },
  {
   "cell_type": "raw",
   "id": "0f085c8c-8edf-4250-a7ea-e39b2a3f43f2",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "raw_mimetype": "text/restructuredtext",
    "tags": []
   },
   "source": [
    ".. index:: Reader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2b56e47-c04f-42ba-bd45-83fd5a6309a4",
   "metadata": {},
   "source": [
    "## The Reader Object"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5445344-9789-4db4-ba60-e55292921eba",
   "metadata": {},
   "source": [
    "The ``Reader`` class provides an access point to further details.\n",
    "\n",
    "For example, by creating a reader instance it’s possible to see how many frames are encoded in ODB-2 data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "32628ee9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "sys.path.insert(0, os.path.abspath('../../..'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b8f9c4ca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "frame(s): 1\n"
     ]
    }
   ],
   "source": [
    "import pyodc as odc\n",
    "\n",
    "r = odc.Reader('example-1.odb')\n",
    "\n",
    "print('frame(s):', len(r.frames))"
   ]
  },
  {
   "cell_type": "raw",
   "id": "e613493c-8c58-428a-a4ef-afd8f00be630",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "raw_mimetype": "text/restructuredtext",
    "tags": []
   },
   "source": [
    ".. index:: Frame"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff3c1ec2-c578-42ca-892f-026b714eeb7f",
   "metadata": {},
   "source": [
    "## The Frame Object"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7ede9b3-0c35-47cc-a272-816cacd7d133",
   "metadata": {},
   "source": [
    "The ``Frame`` class describes specific frames in the stream.\n",
    "\n",
    "For example, to check how many rows and columns are present in particular frames, we can inspect their properties:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5456a2ee-10f4-45f4-80fa-f02b3aab2354",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "row(s):    [10]\n",
      "column(s): [7]\n"
     ]
    }
   ],
   "source": [
    "print('row(s):   ', [f.nrows for f in r.frames])\n",
    "print('column(s):', [f.ncolumns for f in r.frames])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1c7b56d-3202-411d-87dc-6c0240adf18b",
   "metadata": {},
   "source": [
    "Next, we can inspect which columns are present in the frame and their types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d0bf065c-9351-41db-8496-d6456cc83f71",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "column types: [expver:integer, date@hdr:integer, statid@hdr:string, wigos@hdr:string, obsvalue@body:double, integer_missing:integer, double_missing:double]\n"
     ]
    }
   ],
   "source": [
    "print('column types:', *[f.columns for f in r.frames])"
   ]
  },
  {
   "cell_type": "raw",
   "id": "b42f9d25-7b00-43b0-8f29-f165bf70b3b1",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "raw_mimetype": "text/restructuredtext",
    "tags": []
   },
   "source": [
    ".. index:: Column Properties"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a102f16a-ca06-48bc-99c9-1ff9780e1cc9",
   "metadata": {},
   "source": [
    "## Column Properties"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ea31667-afbd-4f67-b66b-49c12f18d32b",
   "metadata": {},
   "source": [
    "The ``ColumnInfo`` class abstracts each column of a frame. To access its properties, simply refer to a particular column in a frame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "692c5e8a-dfff-441f-8938-7f1d2744870d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "column name:      obsvalue@body\n",
      "column data type: DataType.DOUBLE\n",
      "column index:     4\n",
      "column data size: 8\n"
     ]
    }
   ],
   "source": [
    "obsvalue = r.frames[0].columns[4]\n",
    "print('column name:     ', obsvalue.name)\n",
    "print('column data type:', obsvalue.dtype)\n",
    "print('column index:    ', obsvalue.index)\n",
    "print('column data size:', obsvalue.datasize)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ea6991b-2a8d-4cbc-99e0-621612e0ec9b",
   "metadata": {
    "raw_mimetype": "text/restructuredtext",
    "tags": []
   },
   "source": [
    "### Bitfield Columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62a05be3-5939-49d5-89e9-007cb658881d",
   "metadata": {},
   "source": [
    "Bitfield columns provide additional metadata, including flag names, their offsets and sizes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bc923ce7-425b-467e-84f2-dd920729e096",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "flag names:    ['flag_a', 'flag_b', 'flag_c']\n",
      "flag sizes:    [1, 2, 5]\n",
      "flag offsets:  [0, 1, 3]\n"
     ]
    }
   ],
   "source": [
    "r_bf = odc.Reader('example-6.odb')\n",
    "\n",
    "bitfield_column = r_bf.frames[0].columns[0]\n",
    "print('flag names:   ', [flag.name for flag in bitfield_column.bitfields])\n",
    "print('flag sizes:   ', [flag.size for flag in bitfield_column.bitfields])\n",
    "print('flag offsets: ', [flag.offset for flag in bitfield_column.bitfields])"
   ]
  },
  {
   "cell_type": "raw",
   "id": "8049139b-6abe-4b09-bd87-66efea0ce7f5",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "raw_mimetype": "text/restructuredtext",
    "tags": []
   },
   "source": [
    ".. index:: Low-level; Decoding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea0aa8d7-1eea-4ca3-9edf-3af3b687f4ce",
   "metadata": {},
   "source": [
    "## Decoding Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "826ee4d9-770f-4592-a6c7-6f5938bbc43b",
   "metadata": {},
   "source": [
    "Data can also be decoded directly at the frame level, by calling member function ``dataframe`` on a specific frame object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5e717ac5-e91d-4a49-9253-b6e238bfa7a2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   expver  date@hdr statid@hdr        wigos@hdr  obsvalue@body  \\\n",
      "0       1  20210420     stat00  0-12345-0-67890         0.0000   \n",
      "1       1  20210420     stat01  0-12345-0-67891        12.3456   \n",
      "2       1  20210420     stat02  0-12345-0-67892        24.6912   \n",
      "3       1  20210420     stat03  0-12345-0-67893        37.0368   \n",
      "4       1  20210420     stat04  0-12345-0-67894        49.3824   \n",
      "5       1  20210420     stat05  0-12345-0-67895        61.7280   \n",
      "6       1  20210420     stat06  0-12345-0-67896        74.0736   \n",
      "7       1  20210420     stat07  0-12345-0-67897        86.4192   \n",
      "8       1  20210420     stat08  0-12345-0-67898        98.7648   \n",
      "9       1  20210420     stat09  0-12345-0-67899       111.1104   \n",
      "\n",
      "   integer_missing  double_missing  \n",
      "0           1234.0           12.34  \n",
      "1           4321.0           43.21  \n",
      "2              NaN             NaN  \n",
      "3           1234.0           12.34  \n",
      "4           4321.0           43.21  \n",
      "5              NaN             NaN  \n",
      "6           1234.0           12.34  \n",
      "7           4321.0           43.21  \n",
      "8              NaN             NaN  \n",
      "9           1234.0           12.34  \n"
     ]
    }
   ],
   "source": [
    "df_decoded = r.frames[0].dataframe()\n",
    "print(df_decoded)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
