-
Notifications
You must be signed in to change notification settings - Fork 196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Add setExternal in ManagedBytes #1582
Conversation
* Read vector from a connection. | ||
* return true iff a vector was read correctly |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
void ManagedBytes::setExternal(const char *extData, size_t len) | ||
{ | ||
if (!extData) | ||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use the style used in the rest of the code if (...) {
* @param extData, pointer to the external data. | ||
* @param len, num of elements of the external vector | ||
*/ | ||
void setExternal (const char* extData, size_t len) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra space after the function name
} | ||
clear(); | ||
owned = false; | ||
Bytes extb(const_cast<char*>(extData), len); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering why setExternal accepts const
and if this is safe...
The Bytes b
is not const
, therefore the const char
can be modified later by the user by using Vector
methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may be actually the trieckiest part of this PR. The Vector
does not check if it owns the memory (with e.g. a bool owned
like ManagedBytes
) and in fact it should not do it. We must understand whether there is a risk of memory modification or not when the memory is not owned.
May you also please the extra spaces in ManagedBytes::operator=()? |
} | ||
clear(); | ||
owned = false; | ||
Bytes extb(const_cast<char*>(extData), len); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may be actually the trieckiest part of this PR. The Vector
does not check if it owns the memory (with e.g. a bool owned
like ManagedBytes
) and in fact it should not do it. We must understand whether there is a risk of memory modification or not when the memory is not owned.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if in an "external" vector the resize method is called? Note that this will happen also when a vector of dimension different from the current one is read in the read
method.
In general, what is the rationale for this change?
If we need a Vector with external buffer semantics, why do not define a new class VectorView
with this semantics, to make it clear at the type level if the used Vector is owning its own memory or not?
As this new class should not use any dynamic memory allocation, then it could be doable to create a VectorView
on the fly from a yarp;:sig::Vector
or for any other kind of C++ vector (std::vector<double>
, std::array<double, ..>
, Eigen::MatrixXd
) similar (but simpler) to how the Eigen::Map
class works (https://eigen.tuxfamily.org/dox/group__TutorialMapClass.html).
Note: the name VectorView
is inspired from the std::string_view
class that is available since C++17, see https://abseil.io/tips/1 .
cc @S-Dafarra this could be related to our discussion of yesterday.
Related old discussions and comments:
Additional comment to notify that I modified my review. |
@traversaro The rationale behind this is the same behind the
If a Only operation which do not change the memory allocation are safe to be used. |
I was not aware of this, it sounds like a design flaw in
The problem is that, while this peculiar behaviour has always been the case for
Not sure about the specific case you are refereeing to, but any interface that takes in a |
I don't know the course of history of YARP development, but since there exist a I guess that if we want a clean implementation, we either improve
Why is that? |
I think it is a trade-off between efficiency and robustness. If user wants to be safe, data can be copied instead of calling a
Images do not use
Because when someone else owns the memory, he/she can shrink or free that memory regardless of what we do, and there is no way for us to be notified. The next access will then cause a segfault. Some check can be added on our side, for example to skip the But the user knows if a |
Improving documentation is always good! 😄
If we are clear about the behavior induced by a Anyway, I'm open to either create a new dedicated class or to improve the class in conjunction with |
What about |
a99e254
to
edac4ca
Compare
But the resizing of the vector that owns the memory is not controllable in my opinion.. so if we want this method and then efficiency, we have to accept some risks, as @barbalberto said it is a trade off... |
A well done assert should evaporate when compiled in release mode, therefore if yarp is compiled in release, an application using the vector will not be affected by the assert. Maybe we can have a f2f chat about which precaution measures to adopt. |
I agree 😄 Can you please schedule one? |
@barbalberto @traversaro @drdanz @claudiofantacci what about tomorrow in the late morning ? |
@Nicogene Ok for me, please send a meeting request on Outlook, thanks! |
Only loosely related, but just yesterday I was working with |
edac4ca
to
924eba2
Compare
A comment to notify the partecipants that |
…ction. Does not change the behaviour for standard types, but enables the possibility to implement specific read/write function for template types
924eba2
to
b9ee051
Compare
* @param extData, pointer to the external data. | ||
* @param len, size in bytes of the external blob | ||
*/ | ||
void setExternal(const char* extData, size_t len); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2 comments here.
First of all, this should not be const char*
, but just char, since the data is not owned, but can be modified. Second, I suggest to use const Bytes& ext
instead, like in the other constructor, in order to make the interface more coherent
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point but I tried to be as much coherent as possible to yarp::sig::Image::setExternal
:
yarp/src/libYARP_sig/include/yarp/sig/Image.h
Lines 247 to 252 in 29e9108
/** | |
* Use this to wrap an external image. | |
* Make sure to that pixel type and padding quantum are | |
* synchronized (you can set these in the FlexImage class). | |
*/ | |
void setExternal(const void *data, int imgWidth, int imgHeight); |
What is the best option ? 😅
interesting discussion on setExternal, why not adding a runtime check? I also like @traversaro idea to use dedicated classes for static check, however, it seems difficult to do it now. |
@Nicogene I know it is not entirely the fault of the newly introduced setExternal method, but can you please updatae the documentation of the |
Ok, I will do a cleanup/documentation extra commit! |
@Nicogene I do not know if you are still interested in implementing the However, I discovered that a
As we did in the past for similar classes (such as |
For now it is in standby but I will take a look, thanks! |
This PR has to be finished and rebased, and have to take account to the (possible) introduction of move semantics in The move semantics + the shared ownership of the memory are an exploding mix if not managed correctly. I experienced it on my skin with the |
@Nicogene as far as I understand the consensus on this was to drop the If the consensus changed or in any case the decision of adding |
@traversaro if I do remember correctly, we accord to define a separate class called |
As far as I understand, By the way, I just realized: if we want to use |
You are right, Then this PR triggered a nice discussion about memory management and ownership but I think that can be closed now, and open a different one that introduce |
I would say yes, but I have not a strong opinion on that. |
This PR add
⚠️ wait #1657 befor merging
setExternal
methods inyarp::os::ManagedBytes
for the future classVectorMap
.Please review code.