-
Notifications
You must be signed in to change notification settings - Fork 55
Implement OASIS file writer #674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Points to address:
- Write OASIS I/O code in C++. Name it
modmesh::OasisDevice. Wrap it to Python for testing. - Manually verify the written file by loading it using a third-party viewer and paste the screenshot.
modmesh/plot/plane_layer.py
Outdated
| # POSSIBILITY OF SUCH DAMAGE. | ||
|
|
||
|
|
||
| class ManhattanDeltaCodec: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Write I/O code in C++. Python is too slow to be taken seriously for I/O.
Name it modmesh::OasisDevice for now.
tests/test_plane_layer.py
Outdated
| from modmesh.plot import plane_layer | ||
|
|
||
|
|
||
| class ManhattanDeltaCodecTC(unittest.TestCase): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unit tests for the OASIS I/O should be in Python.
8a2d55c to
eb97718
Compare
|
Hi @yungyuc,
Done.
I'm concern that it will have too much diff in this PR if we want to output OASIS file. Althought we have geometry record now, we don't have some necessary OASIS header like magic bytes, START and END. I would like to confirm that this PR should support OASIS file output or we can just test Manhatten 1-delta work properly with unit tests? |
|
Linter did not pass. Please fix code format and then request for review again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some very basic points to address:
- Do not make an intrusive alias in global namespace and for STL things like
std::vector<uint8)_t>. - Rule of 5 for
OasisDevice. - Name as
OasisDevice::write, notwriter, which is not a verb.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| #include <cstdint> | ||
| #include <vector> | ||
|
|
||
| using bytes = std::vector<uint8_t>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not make an intrusive alias like this. It is horrible to maintain.
There is nothing wrong with using std::vector<uint8)_t>? Just spell out the type.
It is horrible to add an alias in the global name space. Unacceptable almost always.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| static void append_1_delta(bytes & segment, int value); | ||
|
|
||
| public: | ||
| OasisDevice() = default; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rule of 5.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
|
|
||
| public: | ||
| OasisDevice() = default; | ||
| static bytes writer(std::vector<int32_t> & values); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Name it write, not writer, which is not a verb.
|
@tigercosmos @KHLee529 please help review |
71b53ff to
d3e62a3
Compare
| namespace python | ||
| { | ||
|
|
||
| using namespace modmesh::oasis; // NOLINT(google-build-using-namespace) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend remove this. Namespace is important for reading the code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. modmesh::oasis namespace is not necessary and should be removed.
And, never using namespace unless it is in a very limited scope and can resolve extreme pain.
cpp/modmesh/oasis/oasis_device.cpp
Outdated
|
|
||
| void OasisDevice::append_1_delta(std::vector<uint8_t> & segment, int value) | ||
| { | ||
| int dir_bit = value < 0 ? 1 : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add const for all the places if applicable.
| namespace oasis | ||
| { | ||
|
|
||
| class OasisDevice |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add document comments for classes and functions.
cpp/modmesh/oasis/oasis_device.cpp
Outdated
| } | ||
| } | ||
|
|
||
| std::vector<uint8_t> OasisDevice::write(std::vector<int32_t> & values) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const std::vector<int32_t> & values
cpp/modmesh/oasis/oasis_device.cpp
Outdated
| namespace oasis | ||
| { | ||
|
|
||
| void OasisDevice::append_1_delta(std::vector<uint8_t> & segment, int value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be a local function in cpp only.
| while (payload >= 1) | ||
| { | ||
| int first_bit = payload >= 128 ? 1 : 0; | ||
| segment.push_back((first_bit << 7) | (payload % 128)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add comments to explain the algorithm. Including source doucment is even better.
modmesh/plot/plane_layer.py
Outdated
| # POSSIBILITY OF SUCH DAMAGE. | ||
|
|
||
|
|
||
| class ManhattanDeltaCodec: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove it.
tests/test_oasis_device.py
Outdated
| device = modmesh.OasisDevice() | ||
| seg = device.writer([+340, +200, -340, -40, +300, -120, -300]) | ||
|
|
||
| self.assertEqual(seg, [0xD0, 0x0A, 0xA0, 0x06, 0xD2, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will be nice if you can explain what you try to test in the comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use code comments to explain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Points to address:
- Do not use namespace
modmesh::oasis. - Add very concise code comments for the key points in unit tests.
| namespace python | ||
| { | ||
|
|
||
| using namespace modmesh::oasis; // NOLINT(google-build-using-namespace) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. modmesh::oasis namespace is not necessary and should be removed.
And, never using namespace unless it is in a very limited scope and can resolve extreme pain.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| namespace modmesh | ||
| { | ||
|
|
||
| namespace oasis |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not use namespace modmesh::oasis.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@c1ydehhx You forgot to fix it.
tests/test_oasis_device.py
Outdated
| device = modmesh.OasisDevice() | ||
| seg = device.writer([+340, +200, -340, -40, +300, -120, -300]) | ||
|
|
||
| self.assertEqual(seg, [0xD0, 0x0A, 0xA0, 0x06, 0xD2, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use code comments to explain.
948efe1 to
c7cec2a
Compare
|
Hi @yungyuc and @tigercosmos. Thanks for the review. I complete OASIS device to store polygon and rectangle and write as OASIS format. The OASIS format will need magic-byte, START record, END record, CELLNAME record and CELL record to work. For each record, I add comment to describe its format and how to find the record spec in OASIS spec draft. I also add some comment to describe how unsigned and signed value work in OASIS. Please refer the comment in Also, the import modmesh
device = modmesh.OasisDevice()
rec_poly = modmesh.OasisPolyRecord([
[70, 720], [410, 720], [410, 920], [70, 920],
[70, 880], [370, 880], [370, 760], [70, 760]])
rec_rect = modmesh.OasisRectRecord([70, 800], 180, 40)
device.add_poly_record(rec_poly)
device.add_rect_record(rec_rect)
rec_poly = modmesh.OasisPolyRecord([
[70, 980], [410, 980], [410, 1180], [70, 1180],
[70, 1140], [370, 1140], [370, 1020], [70, 1020]])
rec_rect = modmesh.OasisRectRecord([70, 1060], 180, 40)
device.add_poly_record(rec_poly)
device.add_rect_record(rec_rect)
with open("test.oas", "wb") as f:
f.write(bytearray(device.write()))It will generate
Please review it. Thanks. |
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| class PolyRecord | ||
| { | ||
| private: | ||
| std::vector<std::pair<int, int>> vertexes; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All non-POD class member data need to prefix m_.
Put member data after member functions like other modmesh classes do. Follow convention unless you have a wonderful reason not to.
Use "vertices" as the plural form of "vertex". Do not use vertexes.
I dislike std::vector member data. But it can be here for a while. They should be changed to a pad soon.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| namespace modmesh | ||
| { | ||
|
|
||
| namespace oasis |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@c1ydehhx You forgot to fix it.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| void add_poly_record(const PolyRecord & record); | ||
| void add_rect_record(const RectRecord & record); | ||
|
|
||
| std::vector<uint8_t> write(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If returning a byte container, rename write to to_bytes.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| * \class PolyRecord | ||
| * \brief Convert Rect information to OASIS polygon record bytes. | ||
| */ | ||
| class PolyRecord |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename PolyRecord to OasisRecordPoly.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| * \class RectRecord | ||
| * \brief Convert Rect information to OASIS rectangle record bytes. | ||
| */ | ||
| class RectRecord |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename RectRecord to OasisRecordRect.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| namespace oasis | ||
| { | ||
|
|
||
| static void append_signed_integer(std::vector<uint8_t> & segment, int value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After removing the namespace modmesh::oasis, move these free functions to OasisDevice as static functions..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You forgot to move the free functions.
tests/test_oasis_device.py
Outdated
| import modmesh | ||
|
|
||
|
|
||
| class OasisRectRecordTC(unittest.TestCase): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good one. Need to be renamed with the C++ class.
tests/test_oasis_device.py
Outdated
| self.assertEqual(record_bytes, list(map(ord, expected_record_bytes))) | ||
|
|
||
|
|
||
| class OasisPolyRecordTC(unittest.TestCase): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good one. Need to be renamed with the C++ class.
tests/test_oasis_device.py
Outdated
| self.assertEqual(record_bytes, list(map(ord, expected_record_bytes))) | ||
|
|
||
|
|
||
| # For the format of OASIS, please refer OasisDevice comment in oasis_device.cpp |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make it clear with the namespace modmesh::OasisDevice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Do not include any Python-related (pybind11 included) headers outside a
pymod/directory.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| */ | ||
|
|
||
| #include <modmesh/base.hpp> | ||
| #include <pybind11/stl.h> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not import any Python-related (pybind11 included) headers outside a pymod/ directory.
47071d4 to
51e49fc
Compare
|
Hi @yungyuc. Thanks for the review. I already fix all issues in comment. I adjust the function name, class name, member name, and comment that mention the class name,. Moreover, I remove pybind header that include in wrong place and remove the For the comment that move free function to Please review it. Thanks. |
You can. Move // Forward declaration (but do not add comment in your PR because it's obvious.
OasisRecordPoly;
OasisRecordRect;
class OasisDevice
{
public:
// all static functions for OasisRecord*.
private:
std::vector<OasisRecordPoly> //...;
std::vector<OasisRecordRect> //...;
};
class OasisRecordPoly
{
// content.s
};
class OasisRecordRect
{
// content.s
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Points to address:
- Split the field
OasisRecordRect::m_bottom_leftto bem_lowerandm_left. - Add end marker to all classes. You forgot some.
- Missing rule of five.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| static void append_end_record_byte(std::vector<uint8_t> & segment); | ||
|
|
||
| template <typename T> | ||
| static void append_record_bytes( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's too short to use two lines. Use a single line for the signature.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| std::vector<uint8_t> to_bytes() const; | ||
|
|
||
| private: | ||
| std::pair<int, int> m_bottom_left; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Split the field m_bottom_left to be m_lower and m_left.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
|
|
||
| private: | ||
| std::pair<int, int> m_bottom_left; | ||
| int w; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
m_ prefix. Please review carefully.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| std::pair<int, int> m_bottom_left; | ||
| int w; | ||
| int h; | ||
| }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Class end marker.
| public: | ||
| explicit OasisRecordPoly(std::vector<std::pair<int, int>> vertices) | ||
| : m_vertices(std::move(vertices)) {}; | ||
| OasisRecordPoly(OasisRecordPoly const &) = default; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You forgot to delete default constructor.
| , h(h) | ||
| { | ||
| } | ||
| OasisRecordRect(OasisRecordRect const &) = default; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
51e49fc to
17db390
Compare
17db390 to
fca59ca
Compare
|
Hi @yungyuc. Thanks for the review. I fix all issue mentioned in comment. I delete copy assignment on Please review it. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You have previous points unaddressed:
- You forgot to move free functions.
- Missing rule of five.
acd4ac1 to
150a8c9
Compare
150a8c9 to
f2dffb9
Compare
|
Hi @yungyuc. Thanks for the review. I just move the free function into std::vector<uint8_t> OasisRecordRect::to_bytes(){
// Implementation ...
OasisDevice::append_unsigned_integer(segment, m_w);
OasisDevice::append_unsigned_integer(segment, m_h);
OasisDevice::append_signed_integer(segment, m_left);
OasisDevice::append_signed_integer(segment, m_lower);
}Also, I delete default constructor in std::vector<OasisRecordPoly> m_polygon_records;
std::vector<OasisRecordRect> m_rect_records;After think a while, since I don't pre-allocate the space to let the vector using default constructor to create element, it still work properly. I already fixed these issue. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I delete default constructor in
OasisRecord*. I previous concern that delete default constructor will happen compilation error since the vector withOasisRecord*can not be construct.std::vector<OasisRecordPoly> m_polygon_records; std::vector<OasisRecordRect> m_rect_records;After think a while, since I don't pre-allocate the space to let the vector using default constructor to create element, it still work properly.
I already fixed these issue. Thanks.
If you want to keep the default constructor, make it explicit like:
OasisRecordPoly() = default;Otherwise it's deleted implicitly: https://stackoverflow.com/a/38995677.
The point is to make your intention explicit. You did not seem to know that not supplying the default constructor actually removed it. It will confuse you and maintainers in the future.
- Review if you want to have the default constructor of
OasisRecord*classes or not. If yes, change the code. If not, leave a comment and stay with the deletion.
Other points to address:
- Move the static functions to the beginning of the class because they do not depend on other code in it.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| void add_poly_record(const OasisRecordPoly & record); | ||
| void add_rect_record(const OasisRecordRect & record); | ||
|
|
||
| static void append_signed_integer(std::vector<uint8_t> & segment, int value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I usually move the static functions to the beginning of the class because they do not depend on other code in it.
It also makes sense to move the static member functions to be before other member functions, because the latter (or the code in the latter) may call the former.
My order is usually: (1) aliases, constants, and enum, (2) static member functions, (3) member functions, (4) member data. It's also the convention used in modmesh.
Please follow the convention.
cpp/modmesh/oasis/oasis_device.hpp
Outdated
| std::vector<uint8_t> to_bytes() const; | ||
|
|
||
| private: | ||
| int m_left; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may use 4 lines for the 4 fields or 1 line for them.
|
Hi @yungyuc. Thanks for the review. I already fix the function order of class. If my realize correct, the member function may include constructor, so I put static function before constructor. I would like to check is it fit the convention. Thanks. For the private static function, I'm not sure the convention that also need to move to the beginning of the class. If so, it will look like this: class OasisDevice
{
public:
static void append_signed_integer(std::vector<uint8_t> & segment, int value);
static void append_unsigned_integer(std::vector<uint8_t> & segment, int value);
private:
static void append_magic_bytes(std::vector<uint8_t> & segment);
static void append_start_record_bytes(std::vector<uint8_t> & segment);
static void append_cell_and_cell_name_record_byte(std::vector<uint8_t> & segment);
static void append_end_record_byte(std::vector<uint8_t> & segment);
public:
// Public member function.
private:
// Private member function.
// Private member.
} /* end class OasisDevice */Please let me know that I just keep the current version or adjust to the version above. Moreover, I condense rect member to 1 line.
After think of the while, I think I'll keep default constructor deleted. Since the empty record of poly or rect is nonsense afaik, I'll keep the default constructor deleted. Thanks for the instruction about the behavior of constructor. These changes already co-work with Codex to check the point you address is met. |

In this PR, I added OASIS device. This device support writer only currently. This writer implement manhattan delta format by OASIS spec.
For example, if we expected +340 for N/S direction, it will be encoded with
0xA8 0x05, and -340 will be0xA9 0x05.I added some test to make sure it work properly, and I add some edge case to make sure it will not buggy when met edge case.