-
-
Notifications
You must be signed in to change notification settings - Fork 16
Inflection 77: Adding C and C++ API for stating source grammemes and tests for the same. #174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| * @param value - The speakable string to convert to a concept | ||
| * @param intitialConstraints - The intitial constraints for the map. | ||
| */ | ||
| InflectableStringConcept(const SemanticFeatureModel* model, const SpeakableString& value, const std::map<SemanticFeature, ::std::u16string> intitialConstraints); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The map should be a const & for the API.
| * @param speakFeature The speakFeature from the SemanticFeatureModel that represents the SemanticFeature for the speak information for a SpeakableString. | ||
| * @param constraintMap The intitial constraint map. | ||
| */ | ||
| DisplayValue(const SpeakableString& value, const SemanticFeature& speakFeature, const ::std::map<SemanticFeature, ::std::u16string> constraintMap); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The map should be a const & for the API.
| DisplayValue::DisplayValue(const SpeakableString& value, const SemanticFeature& speakFeature) | ||
| : DisplayValue(value.getPrint(), {}) | ||
| DisplayValue::DisplayValue( | ||
| const SpeakableString& value, | ||
| const SemanticFeature& speakFeature, | ||
| const ::std::map<SemanticFeature, ::std::u16string> constraintMap) | ||
| : DisplayValue(value.getPrint(), constraintMap) | ||
| { | ||
| if (!value.speakEqualsPrint()) { | ||
| constraintMap.emplace(speakFeature, value.getSpeak()); | ||
| this->constraintMap.emplace(speakFeature, value.getSpeak()); | ||
| } | ||
| } | ||
|
|
||
| DisplayValue::DisplayValue( | ||
| const SpeakableString& value, | ||
| const SemanticFeature& speakFeature) | ||
| : DisplayValue(value.getPrint(), {}) | ||
| { | ||
| if (!value.speakEqualsPrint()) { | ||
| this->constraintMap.emplace(speakFeature, value.getSpeak()); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please declare these functions in the same order as the header? It makes it easier to navigate.
Also the 2 argument constructor with the speakFeature can now call the new constructor with an empty map. Then the redundant if statement can be removed. It's best to keep the copy and paste code to a minimum to improve the maintenance.
| InflectableStringConcept::InflectableStringConcept(const SemanticFeatureModel* model, const SpeakableString& value) | ||
| InflectableStringConcept::InflectableStringConcept( | ||
| const SemanticFeatureModel* model, | ||
| const SpeakableString& value, | ||
| const ::std::map<SemanticFeature, ::std::u16string> intitialConstraints | ||
| ) | ||
| : super(model) | ||
| , value(value) | ||
| , defaultDisplayValue(value, *npc(super::getSpeakFeature()), intitialConstraints) | ||
| { | ||
| } | ||
|
|
||
| InflectableStringConcept::InflectableStringConcept( | ||
| const SemanticFeatureModel* model, | ||
| const SpeakableString& value | ||
| ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please declare these functions in the same order as the header? It makes it easier to navigate.
Also the 2 argument constructor with the SpeakableString can now call the new constructor with an empty map. Then the copied code can be removed. It's best to keep the copy and paste code to a minimum to improve the maintenance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure I would keep that in mind.
Being a fresher I wasn't aware.
|
@grhoten I have added the C API too can you please review and tell me the next steps. |
| * This is set to a failure when a failure has occurred during execution. | ||
| */ | ||
| INFLECTION_CAPI IDInflectableStringConcept* iinf_createWithConstraints(const IDSemanticFeatureModel* model, const IDSpeakableString* value, | ||
| const inflection::dialog::SemanticFeature* features, const char16_t** values, int32_t SemanticFeaturesLen, UErrorCode* status); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
C++ API is not allowed in C API. Please see ipron_createWithDefaults for how to pass in such values. That API also ensures that the 2 lengths are exactly the same.
For consistency with the PronounConcept API, it would be ideal if this was also named iinf_createWithDefaults to be consistent with the naming.
grhoten
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All new API need tests. The code should always be in a releasable state. Please add tests for these changes. The current code should be well over 90% line code coverage.
Please also state which issue is being fixed with these changes. That is best done with prefixing your commits and pull request summary with "Inflection-77".
| #include <inflection/dialog/SemanticFeatureConcept.h> | ||
| #include <inflection/dialog/SemanticFeatureModel.h> | ||
|
|
||
| #include <inflection/dialog/SemanticUtils.hpp> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This internal C++ header should not be used in public C API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will fix that
| if (pronounData->numValues() == 0) { | ||
| throw ::inflection::exception::IllegalArgumentException(u"Display data can not be empty."); | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that this file needs to change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup sure, I mistakenly added a line and forgot
|
@grhoten
|
|
@grhoten
|
static functions are not available outside of the file. It also compiles. So that implies that the changes are OK.
As long as there are a couple of tests, that should be sufficient to show the utility of the API.
I recommend including a test of iinf_createWithDefaults in InflectableStringConceptTest-c.cpp.
|
| <test><source definiteness="definite">cometa</source><initial gender="masculine"/><result number="singular" gender="masculine">el cometa</result></test> | ||
| <test><source definiteness="definite">cometa</source><initial gender="feminine"/><result number="singular" gender="feminine">la cometa</result></test> | ||
| <test><source definiteness="definite">QQQQ</source><initial gender="masculine" number="plural"/><result number="plural" gender="masculine">los QQQQ</result></test> | ||
| <test><source definiteness="definite">QQQQ</source><initial gender="feminine" number="plural"/><result number="plural" gender="masculine">las QQQQ</result></test> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These tests look reasonable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me correct my comment. The structure looks generally good, but the last test is wrong. The initial gender is feminine. So the gender result should be feminine too. This looks like a copy and paste error.
| } | ||
| return new SpeakableString(result); | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@grhoten
Was this the expected behavior in this function? If not, could you please point out where I am going wrong?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. Within the DictionaryLookupFunction, you're only returning the grammeme value. This does not need to change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DictionaryLookupFunction::getFeatureValue likely needs to be updated to query the relevant values from displayValue. It should probably query DisplayValue::getFeatureValue or DisplayValue::getConstraintMap
Okay sure I wanted to know exactly what I need to do, based on your previous comment I came up with this commit, could you please tell what is expected to make the tests pass so that i could implement the same and further add tests in InfectableStringConceptTest-c.cpp as well to complete this task ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please revert the change to this file. It looks like it's only changing the empty lines.
…createWithDefaults as InflectableStringConceptTest-c#testCreateWithDefaults
|
Hello!! |
| auto initialConstraint = initialConstraintMap.find(feature); | ||
| if (initialConstraint != initialConstraintMap.end()) { | ||
| return new SpeakableString(initialConstraint->second); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes are confusing. Why are these changes needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More precisely, why is getFeatureValue being called here? Why is the feature name being checked for gender? What's special about gender or number? Why not grammatical case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
auto displayConstraint = displayConstraintMap.find(feature);
if (displayConstraint != displayConstraintMap.end()) {
auto numberFeature = npc(getModel())->getFeature(u"number");
if (feature.getName() == u"gender" && baseValue && numberFeature != nullptr
&& displayConstraintMap.find(*npc(numberFeature)) != displayConstraintMap.end()) {
return baseValue.release();
}
return new SpeakableString(displayConstraint->second);
} This is the test case which was failing before the changes:
<test>
<source definiteness="definite">cometa</source>
<initial gender="masculine"/>
<result gender="masculine" number="singular">el cometa</result>
</test>-
InflectableStringConcept::getFeatureValueasks the default display function for a renderedDisplayValue, so feature lookups reflect whatever the display layer injected (articles, inflections, etc.). Spanish determiners add bothgenderandnumberto that map, and the gender can contradict the lexeme default (la cometa). -
To avoid reporting the article’s guess as the noun’s gender, the code detects
feature == "gender"and, when the number constraint is present too, returns the lexeme’s base value instead of the display-layer value. English articles never set gender, and no language mutates case in this path yet (I am not sure though), so only the gender/number pair needs a guard today. -
InflectableStringConcept::getFeatureValueasks the default display function for a renderedDisplayValue, so feature lookups reflect whatever the display layer injected (articles, inflections, etc.). -
Spanish determiners add both
genderandnumberto that map, and the gender can contradict the lexeme default (la cometa).
To avoid reporting the article’s guess as the noun’s gender, the code detectsfeature == "gender"and, when the number constraint is present too, returns the lexeme’s base value instead of the display-layer value, as these conditions when satisfied point to the case when we have an article being inserted. -
Grammatical case isn’t treated specially because the Spanish article logic never synthesizes case information—so there’s no regression to guard against. If a language later injected case-altering determiners we’d need a similar safeguard, but today only the gender/number pair exhibits the mismatch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This behavior does not scale, and it does not make sense in my mind. An InflectableStringConcept is just a simplified SemanticConcept. A SemanticConcept should not have this issue. It does not have this kind of logic, and it does not check on specific grammatical category values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you're right it is just a general constructor and logic, I think the underlying locale specific synthesizer needs to be addressed for the failing test case.
|
@grhoten instead to get the include path automatically. |
| add_library(xml2 INTERFACE IMPORTED GLOBAL) | ||
| set_target_properties(xml2 PROPERTIES IMPORTED_LIBNAME xml2) | ||
| target_include_directories(xml2 INTERFACE ${CMAKE_OSX_SYSROOT}/usr/include/libxml2) | ||
| find_package(LibXml2 REQUIRED) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a good improvement.
| #include <inflection/npc.hpp> | ||
|
|
||
| std::map<std::u16string, std::u16string> XMLUtils::getAttributes(xmlNodePtr node) { | ||
| if(node == nullptr) return { }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please split this line and use curly braces. It makes it easier to place break points.
|
The only test failing is this one: <test><source definiteness="definite">QQQQ</source><initial gender="feminine" number="plural"/><result number="plural" gender="masculine">las QQQQ</result></test>This is a spanish-specific issue in which the article has association with gender and the Could you suggest some next-steps? |
|
@grhoten Is the discussion going in the wrong direction or if you could suggest some other way? We only need to address this issue for the failing test |
| auto inflectableConcept = iinf_createWithDefaults(model, | ||
| sourceString, | ||
| defaultConstraints, | ||
| static_cast<int32_t>(sizeof(defaultConstraints) / sizeof(defaultConstraints[0])), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change this calculation to use std::ssize instead. It's a C++20 feature.
|
|
||
| #include <inflection/npc.hpp> | ||
| #include <inflection/dialog/InflectableStringConcept.hpp> | ||
| #include <inflection/dialog/SemanticFeatureConcept.h> | ||
| #include <inflection/exception/ClassCastException.hpp> | ||
| #include <inflection/util/ULocale.hpp> | ||
| #include <inflection/dialog/SemanticFeature.hpp> | ||
| #include <inflection/dialog/SemanticUtils.hpp> | ||
| #include <inflection/dialog/SemanticFeatureModel.hpp> | ||
| #include <inflection/util/TypeConversionUtils.hpp> | ||
| #include <inflection/util/Validate.hpp> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally, I've tried to keep the first line the header for the current file, followed by an empty line, sort the headers, and leave npc.h last.
Can you please sort the headers in this section?
| constraintMap.emplace(*npc(getSpeakFeature()), value.getSpeak()); | ||
| } | ||
| SemanticFeatureModel_DisplayData displayData({DisplayValue(value.getPrint(), constraintMap)}); | ||
| SemanticFeatureModel_DisplayData displayData({defaultDisplayValue}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For future reference to self and other reviewers, the DisplayValue constructor with a SpeakableString takes care of the speak usage. It didn't disappear.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@grhoten
Yes, in the latest commit I did the following change in the InflectableStringConcept::getDisplayValue:
As their are initial constraints as well we might need to pass the constraintMap which was constructed into the defaultDisplayValue. I am not sure, please validate this step, Although the tests are passing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this step is correct i will resolve the macos test issue and remove the lines used for debugging so that the PR is ready to merge.
Sorry for the delay. The changes are in the right direction now. Once the new comments are addressed, then these changes should be ready to merge. |
| * Copyright 2021-2024 Apple Inc. All rights reserved. | ||
| */ | ||
| #include "catch2/catch_test_macros.hpp" | ||
| #include <iostream> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove the logging changes with iostream. Please use Catch for logging any relevant information. The tests should run clean by default so that it's easier to find relevant information for failing tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes actually I just wanted to be sure that the changes I did to InflectableStringConcept::getDisplayValue are correct I mean using the constraintMap from defaultDisplayValue so that we use the initial constraints as well, please validate the overall changes to that function from all commits . And then I will do a final cleanup commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything else looks fine, except for the comments that I provided about an hour ago.
| const auto displayValueResult = getDisplayValue(true); | ||
| if (displayValueResult) { | ||
| return npc(defaultFeatureFunction)->getFeatureValue(*displayValueResult, constraints); | ||
| return npc(defaultFeatureFunction)->getFeatureValue(*displayValueResult, constraints); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need for whitespace after the semicolon.
| // auto constraintMap = defaultDisplayValue.getConstraintMap(); | ||
| // if(constraintMap.find(feature) != constraintMap.end()) { | ||
| // return new SpeakableString(constraintMap[feature]); | ||
| // } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't leave commented out code in your changes. It makes it harder to read.
| <test><source definiteness="definite">cometa</source><initial gender="feminine"/><result number="singular" gender="feminine">la cometa</result></test> | ||
| <test><source definiteness="definite">QQQQ</source><initial gender="masculine" number="plural"/><result number="plural" gender="masculine">los QQQQ</result></test> | ||
| <test><source definiteness="definite">QQQQ</source><initial gender="feminine" number="plural"/><result number="plural" gender="masculine">las QQQQ</result></test> | ||
| <test><source definiteness="definite">QQQQ</source><initial gender="feminine" number="plural"/><result number="plural" gender="feminine">las QQQQ</result></test> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
|
I am not sure about the macos test issue, please look into the same and other than that the code is ready to merge. |
grhoten
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The macOS issue is a separate issue. ICU 78 has different behavior, and it requires the tests to be updated. The curly quote has to be switched to an ASCII straight quote.
We will have to update that in a separate pull request.

@grhoten
Issue #77
Could you please review this so that I could move forward for C API and XML parsing too.