Georg_Brantegger/Python-DT_Slot_3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

validation data sorting and folder cleanup

Browse Source
This commit is contained in:
Georg ´Brantegger
2022-09-20 15:36:25 +02:00
parent c9d64d8011
commit 84cd8fd5f4
25 changed files with 1767572 additions and 1314685 deletions
Download Patch File Download Diff File Expand all files Collapse all files

826465
Validation Data/raw data Untertweng/AL_Oberwasser.txt Normal file
View File

File diff suppressed because it is too large Load Diff

17432
Validation Data/raw data Untertweng/AL_Pegel.txt Normal file
View File

File diff suppressed because it is too large Load Diff

6
Validation Data/raw data Untertweng/INFORMATION.txt Normal file
View File

@@ -0,0 +1,6 @@
Zeitraum UNIX >= 01.06.2022 00:00 Uhr CEST und < 01.09.2022 00:00 Uhr CEST
Daten aus spontanen Archiven aus Datenbank KW_UT (Archive 10,40,50)
Spaltenseparator ;
Dezimaltrennzeichen .
Spalten: Siehe .txt Files
Daten enthalten nur gültige Werte, d.h. Invalid-Bit (18. Bit) im Status ist FALSE

23209
Validation Data/raw data Untertweng/M1_Druck.txt Normal file
View File

File diff suppressed because it is too large Load Diff

37228
Validation Data/raw data Untertweng/M1_LA.txt Normal file
View File

File diff suppressed because it is too large Load Diff

36449
Validation Data/raw data Untertweng/M2_Druck.txt Normal file
View File

File diff suppressed because it is too large Load Diff

373714
Validation Data/raw data Untertweng/M2_LA.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1821
Validation Data/raw data Untertweng/Schnellschluss/2015_08_24 18.00 M1 SS100%.csv Normal file
View File

File diff suppressed because it is too large Load Diff

1285
Validation Data/raw data Untertweng/Schnellschluss/2015_08_25 08.40 M2 SS100%.csv Normal file
View File

File diff suppressed because it is too large Load Diff

6980
Validation Data/raw data Untertweng/Schnellschluss/2015_08_25 15.20 M12 SS100%.csv Normal file
View File

File diff suppressed because it is too large Load Diff

118
Validation Data/raw data Untertweng/consolidate_validation_data_Untertweng.ipynb Normal file
View File

@@ -0,0 +1,118 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"pegel_df = pd.read_csv('AL_Pegel.txt',delimiter=';')\n",
"M1_p_df = pd.read_csv('M1_Druck.txt',delimiter=';')\n",
"M2_p_df = pd.read_csv('M2_Druck.txt',delimiter=';')\n",
"M1_LA_df = pd.read_csv('M1_LA.txt',delimiter=';')\n",
"M2_LA_df = pd.read_csv('M2_LA.txt',delimiter=';')"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"pegel_df['Timestamp'] = pegel_df['TIMESTAMP UNIX']+pegel_df['TIMESTAMP MS']/1000.\n",
"M1_p_df['Timestamp'] = M1_p_df['TIMESTAMP UNIX']+M1_p_df['TIMESTAMP MS']/1000.\n",
"M2_p_df['Timestamp'] = M2_p_df['TIMESTAMP UNIX']+M2_p_df['TIMESTAMP MS']/1000.\n",
"M1_LA_df['Timestamp'] = M1_LA_df['TIMESTAMP UNIX']+M1_LA_df['TIMESTAMP MS']/1000.\n",
"M2_LA_df['Timestamp'] = M2_LA_df['TIMESTAMP UNIX']+M2_LA_df['TIMESTAMP MS']/1000."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"pegel_df.set_index('Timestamp',inplace=True)\n",
"M1_p_df.set_index('Timestamp',inplace=True)\n",
"M2_p_df.set_index('Timestamp',inplace=True)\n",
"M1_LA_df.set_index('Timestamp',inplace=True)\n",
"M2_LA_df.set_index('Timestamp',inplace=True)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"pegel_df.drop(columns=['VARIABLE','TIMESTAMP UNIX', 'TIMESTAMP MS'],inplace=True)\n",
"M1_p_df.drop(columns=['VARIABLE','TIMESTAMP UNIX', 'TIMESTAMP MS'],inplace=True)\n",
"M2_p_df.drop(columns=['VARIABLE','TIMESTAMP UNIX', 'TIMESTAMP MS'],inplace=True)\n",
"M1_LA_df.drop(columns=['VARIABLE','TIMESTAMP UNIX', 'TIMESTAMP MS'],inplace=True)\n",
"M2_LA_df.drop(columns=['VARIABLE','TIMESTAMP UNIX', 'TIMESTAMP MS'],inplace=True)\n",
"\n",
"pegel_df.rename(columns={'VALUE': 'TB-Pegel'},inplace=True)\n",
"M1_p_df.rename(columns={'VALUE': 'M1-Druck'},inplace=True)\n",
"M2_p_df.rename(columns={'VALUE': 'M2-Druck'},inplace=True)\n",
"M1_LA_df.rename(columns={'VALUE': 'M1-LA'},inplace=True)\n",
"M2_LA_df.rename(columns={'VALUE': 'M2-LA'},inplace=True)\n"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"UT_df = pegel_df.join([M1_LA_df,M1_p_df,M2_LA_df,M2_p_df],how='outer').sort_index()"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"UT_df.to_csv('UT_df.csv')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.8.13 ('DT_Slot_3')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.13"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "4a28055eb8a3160fa4c7e4fca69770c4e0a1add985300856aa3fcf4ce32a2c48"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}