dquality check does not reproduce the same results.

[decarlof]

Running multiple time python doc/demo/dquality_check.py
does not reproduce the same results.
data file and configs I used are at petrel#tomography/databank/dataExchange/DataQuality/test_01/

Output is:
run 1:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 832.9627619225995, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1372.6562367713757, 2, False)

run2:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 239.71817869298599, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1358.8717235151589, 2, False)

run 3:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)

run 4:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)

Here is full output:
txmtwo% python doc/demo/dquality_check.py
Verification type: hdf_structure
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta
- units
the following tags are missing:
- /exchange/missing
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 832.9627619225995, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1372.6562367713757, 2, False)
finished
txmtwo% python doc/demo/dquality_check.py
Verification type: hdf_structure
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta
- units
the following tags are missing:
- /exchange/missing
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 239.71817869298599, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1358.8717235151589, 2, False)
finished
txmtwo% python doc/demo/dquality_check.py
Verification type: hdf_structure
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta
- units
the following tags are missing:
- /exchange/missing
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)
finished
txmtwo% python doc/demo/dquality_check.py
Verification type: hdf_structure
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta
- units
the following tags are missing:
- /exchange/missing
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)
finished

[bfrosik]

I will run the test and try to figure this out.

Barbara

---

From: Francesco De Carlo [notifications@github.com]
Sent: Tuesday, March 08, 2016 11:15 PM
To: bfrosik/data-quality
Subject: [data-quality] dquality check does not reproduce the same results. (#10)

Running multiple time python doc/demo/dquality_check.py
does not reproduce the same results.
data file and configs I used are at petrel#tomography/databank/dataExchange/DataQuality/test_01/

Output is:
run 1:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 832.9627619225995, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1372.6562367713757, 2, False)

run2:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 239.71817869298599, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1358.8717235151589, 2, False)

run 3:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)

run 4:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)

Here is full output:
txmtwo% python doc/demo/dquality_check.py
Verification type: hdf_structure
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta

• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 832.9627619225995, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1372.6562367713757, 2, False)
  finished
  txmtwo% python doc/demo/dquality_check.py
  Verification type: hdf_structure
  The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
  The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
  incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
  incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
  the following attributes are missing in tag /exchange/theta
• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 239.71817869298599, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1358.8717235151589, 2, False)
  finished
  txmtwo% python doc/demo/dquality_check.py
  Verification type: hdf_structure
  The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
  The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
  incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
  incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
  the following attributes are missing in tag /exchange/theta
• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)
  finished
  txmtwo% python doc/demo/dquality_check.py
  Verification type: hdf_structure
  The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
  The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
  incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
  incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
  the following attributes are missing in tag /exchange/theta
• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)
  finished

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/10.

[bfrosik]

Hi Francesco,

I can't reproduce the problem. I am running it with the config and schemas from the petrel, and every time I run I got these results:

bash-4.1$ python dquality_check.py

---

hdf_structure

---

Verification type: hdf_structure

---

/home/phoebus/BFROSIK/dqschemas/basicHDF.json

---

test.h5
/home/phoebus/BFROSIK/dqschemas/basicHDF.json
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta
- units
the following tags are missing:
- /exchange/missing
('result: file name, result, quality id, error: ', 'test.h5', 21.160364711985867, 1, False)
('result: file name, result, quality id, error: ', 'test.h5', 54.066940477190499, 2, False)
finished

Maybe we can try to rerun it during the meeting?

Thank you,
Barbara

---

From: Francesco De Carlo [notifications@github.com]
Sent: Tuesday, March 08, 2016 11:15 PM
To: bfrosik/data-quality
Subject: [data-quality] dquality check does not reproduce the same results. (#10)

Running multiple time python doc/demo/dquality_check.py
does not reproduce the same results.
data file and configs I used are at petrel#tomography/databank/dataExchange/DataQuality/test_01/

Output is:
run 1:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 832.9627619225995, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1372.6562367713757, 2, False)

run2:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 239.71817869298599, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1358.8717235151589, 2, False)

run 3:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)

run 4:
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)

Here is full output:
txmtwo% python doc/demo/dquality_check.py
Verification type: hdf_structure
The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
the following attributes are missing in tag /exchange/theta

• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 832.9627619225995, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1372.6562367713757, 2, False)
  finished
  txmtwo% python doc/demo/dquality_check.py
  Verification type: hdf_structure
  The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
  The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
  incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
  incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
  the following attributes are missing in tag /exchange/theta
• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 239.71817869298599, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 1358.8717235151589, 2, False)
  finished
  txmtwo% python doc/demo/dquality_check.py
  Verification type: hdf_structure
  The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
  The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
  incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
  incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
  the following attributes are missing in tag /exchange/theta
• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)
  finished
  txmtwo% python doc/demo/dquality_check.py
  Verification type: hdf_structure
  The dataset /exchange/data dimension 0 is wrong: it is [721] but should be [722]
  The dataset /exchange/data dimension 2 is wrong: it is [2448] but should be [2449]
  incorrect attribute in /exchange/data_dark: is axes:theta:y:x but should be axes:theta_dark:y:x
  incorrect attribute in /exchange/data_white: is axes:theta:y:x but should be axes:theta_white:y:x
  the following attributes are missing in tag /exchange/theta
• units
  the following tags are missing:
• /exchange/missing
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 465.80563977808737, 1, False)
  ('result: file name, result, quality id, error: ', '/local/data/test/test.h5', 54.066940477190499, 2, False)
  finished

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/10.