README.md 2.78 KB
Newer Older
Fahad Ashraf's avatar
Fahad Ashraf committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
# Elasticsearch Test Task

This is a test task for the collaborative intelligence project WS 2020/2021. Task consists of following:

1. Setting up elasticsearch on a linux
2. Create an index and add a mapping via curl
3. Add the "waterLevelReports.csv" statistics as document to the index
4. Execute a curl request to get an aggregation of one of the fields in the documents


## Installation
### Elasticseach

```bash
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.2-x86_64.rpm

sudo rpm -i elasticsearch-7.9.2-x86_64.rpm

sudo service elasticsearch start
```

## Usage

The main.py python script contains all the functions for the creating index mapping, indexing documents, aggregation and deleting index.

Run the following command on your terminal in this repo:

```bash
python main.py
```

## Curl requests and responses
Here I'm just going to put some curl requests and their responses

### Creating index mappping

```bash
curl -X PUT "localhost:9200/water-levels" -H 'Content-Type: application/json' -d'
{             
  "mappings": {
      "properties": {
        "Messst-Nr": { "type": "long" }, 
        "Messstelle": { "type": "text" },
        "Datum": { "type": "text" },
        "Probeart": { "type": "keyword" },
        "Parameter-Nr": { "type": "text" },
        "Parameter": { "type": "text" },
        "Einheit": { "type": "text" },
        "Status": { "type": "keyword" },
        "Wert": { "type": "long" }
      }
    }
}'
```
Response:
```json
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "water-levels"
}
```

### Indexing a document
```bash
curl -X PUT "localhost:9200/water-levels/_doc/1" -H 'Content-Type: application/json' -d'
{             
  "Messst-Nr": "2546188600",
  "Messstelle": "3039 Kaiserslautern, Erfenbach",
  "Datum": "13.11.1978",
  "Probeart": "Messung - GW-Stände-/Quellschüttungen",
  "Parameter-Nr": "09030/00",
  "Parameter": "Wasserstand unter Messpunkt",
  "Einheit": "m",
  "Status": "",
  "Wert": "8.91"
}'
```
Response:
```json
{
  "_index": "water-levels",
  "_type": "_doc",
  "_id": "1",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 0,
  "_primary_term": 1
}
```

### Get an aggregation

```bash
curl -X GET "localhost:9200/water-levels/_search" -H 'Content-Type: application/json' -d'
{
  "size": 0,
  "aggs": {
    "wert_avg": {
      "avg": {
        "field": "Wert"
      }
    }
  }
}'
```
Response:

```json
{
  "took": 883,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2136,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "wert_avg": {
      "value": 7.79696394686907
    }
  }
}
```