MarkLogic : Design question on Query Options Vs Transform
Our team creates a bunch of custom REST APIs (v1/resources/...) and expose them as enterprise services to other stakeholders, who do not need to know anything about MarkLogic. However, our team is responsible for creating, enhancing and maintaining the server-side scripting (we use JavaScript) within MarkLogic.
While creating custom REST APIs, our current design to meet the search requirements is to start with a Query Options, incorporate as many requirements in Query options, and for any requirements that could not be met by Query Options (for example, sorting within a document, complex XPath, merge with other documents etc.), code within the Java Script extension program (technically not a transform but conceptually similar to a transform).
With the limitations in Query options, increasingly, most of our logic is going into the Javascript extension programs and the query options seem to be just a maintenance overhead. Do we really need to maintain a query options file for every REST extension, while the transforms offer much powerful functionality? Can I get rid of Query options and just use server side Java Script code (conceptually, similar to transforms)? Initially, our thought was that Query options is configuration based and hence changing query options is not exactly a code change, however, based on our experience, we realized that changing query options also involves deployment, regression testing and all other activities. Hence I do not see any specific advantage of query options, in our case (creating custom REST APIs).
Design gurus, please suggest!
marklogic-9 marklogic-dhf
asked Nov 22 at 18:39
Bhanu
546
add a comment |
Our team creates a bunch of custom REST APIs (v1/resources/...) and expose them as enterprise services to other stakeholders, who do not need to know anything about MarkLogic. However, our team is responsible for creating, enhancing and maintaining the server-side scripting (we use JavaScript) within MarkLogic.
While creating custom REST APIs, our current design to meet the search requirements is to start with a Query Options, incorporate as many requirements in Query options, and for any requirements that could not be met by Query Options (for example, sorting within a document, complex XPath, merge with other documents etc.), code within the Java Script extension program (technically not a transform but conceptually similar to a transform).
With the limitations in Query options, increasingly, most of our logic is going into the Javascript extension programs and the query options seem to be just a maintenance overhead. Do we really need to maintain a query options file for every REST extension, while the transforms offer much powerful functionality? Can I get rid of Query options and just use server side Java Script code (conceptually, similar to transforms)? Initially, our thought was that Query options is configuration based and hence changing query options is not exactly a code change, however, based on our experience, we realized that changing query options also involves deployment, regression testing and all other activities. Hence I do not see any specific advantage of query options, in our case (creating custom REST APIs).
Design gurus, please suggest!
marklogic-9 marklogic-dhf
asked Nov 22 at 18:39
Bhanu
546
add a comment |
Our team creates a bunch of custom REST APIs (v1/resources/...) and expose them as enterprise services to other stakeholders, who do not need to know anything about MarkLogic. However, our team is responsible for creating, enhancing and maintaining the server-side scripting (we use JavaScript) within MarkLogic.
While creating custom REST APIs, our current design to meet the search requirements is to start with a Query Options, incorporate as many requirements in Query options, and for any requirements that could not be met by Query Options (for example, sorting within a document, complex XPath, merge with other documents etc.), code within the Java Script extension program (technically not a transform but conceptually similar to a transform).
With the limitations in Query options, increasingly, most of our logic is going into the Javascript extension programs and the query options seem to be just a maintenance overhead. Do we really need to maintain a query options file for every REST extension, while the transforms offer much powerful functionality? Can I get rid of Query options and just use server side Java Script code (conceptually, similar to transforms)? Initially, our thought was that Query options is configuration based and hence changing query options is not exactly a code change, however, based on our experience, we realized that changing query options also involves deployment, regression testing and all other activities. Hence I do not see any specific advantage of query options, in our case (creating custom REST APIs).
Design gurus, please suggest!
marklogic-9 marklogic-dhf
asked Nov 22 at 18:39
Bhanu
546
Our team creates a bunch of custom REST APIs (v1/resources/...) and expose them as enterprise services to other stakeholders, who do not need to know anything about MarkLogic. However, our team is responsible for creating, enhancing and maintaining the server-side scripting (we use JavaScript) within MarkLogic.
While creating custom REST APIs, our current design to meet the search requirements is to start with a Query Options, incorporate as many requirements in Query options, and for any requirements that could not be met by Query Options (for example, sorting within a document, complex XPath, merge with other documents etc.), code within the Java Script extension program (technically not a transform but conceptually similar to a transform).
With the limitations in Query options, increasingly, most of our logic is going into the Javascript extension programs and the query options seem to be just a maintenance overhead. Do we really need to maintain a query options file for every REST extension, while the transforms offer much powerful functionality? Can I get rid of Query options and just use server side Java Script code (conceptually, similar to transforms)? Initially, our thought was that Query options is configuration based and hence changing query options is not exactly a code change, however, based on our experience, we realized that changing query options also involves deployment, regression testing and all other activities. Hence I do not see any specific advantage of query options, in our case (creating custom REST APIs).
Design gurus, please suggest!
marklogic-9 marklogic-dhf
marklogic-9 marklogic-dhf
asked Nov 22 at 18:39
Bhanu
546
asked Nov 22 at 18:39
Bhanu
546
asked Nov 22 at 18:39
Bhanu
546
asked Nov 22 at 18:39
Bhanu
546
asked Nov 22 at 18:39
Bhanu
546
546
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
These questions are tough to answer without properly knowing the whole situation. You talk about custom REST endpoints, but are using REST extensions on the built-in REST api for that purpose. Are you using anything else of the built-in REST api? You use search options, but don't use them for /v1/search. Could you have done the same with /v1/search with those options, but an extra REST Transform on top? And you speak of a lot of data mangling happening at sub-document level. Have you considered doing part of that processing upfront, or organize your data differently so that handling at request-time becomes much simpler?
A lot of questions, and a lot of possibilities. It is hard to give one straight answer on such an open question. I hope I gave some food for thought nonetheless.
HTH!
answered Nov 22 at 19:30
grtjn
15.1k11730
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436604%2fmarklogic-design-question-on-query-options-vs-transform%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
These questions are tough to answer without properly knowing the whole situation. You talk about custom REST endpoints, but are using REST extensions on the built-in REST api for that purpose. Are you using anything else of the built-in REST api? You use search options, but don't use them for /v1/search. Could you have done the same with /v1/search with those options, but an extra REST Transform on top? And you speak of a lot of data mangling happening at sub-document level. Have you considered doing part of that processing upfront, or organize your data differently so that handling at request-time becomes much simpler?
A lot of questions, and a lot of possibilities. It is hard to give one straight answer on such an open question. I hope I gave some food for thought nonetheless.
HTH!
answered Nov 22 at 19:30
grtjn
15.1k11730
add a comment |
These questions are tough to answer without properly knowing the whole situation. You talk about custom REST endpoints, but are using REST extensions on the built-in REST api for that purpose. Are you using anything else of the built-in REST api? You use search options, but don't use them for /v1/search. Could you have done the same with /v1/search with those options, but an extra REST Transform on top? And you speak of a lot of data mangling happening at sub-document level. Have you considered doing part of that processing upfront, or organize your data differently so that handling at request-time becomes much simpler?
A lot of questions, and a lot of possibilities. It is hard to give one straight answer on such an open question. I hope I gave some food for thought nonetheless.
HTH!
answered Nov 22 at 19:30
grtjn
15.1k11730
add a comment |
These questions are tough to answer without properly knowing the whole situation. You talk about custom REST endpoints, but are using REST extensions on the built-in REST api for that purpose. Are you using anything else of the built-in REST api? You use search options, but don't use them for /v1/search. Could you have done the same with /v1/search with those options, but an extra REST Transform on top? And you speak of a lot of data mangling happening at sub-document level. Have you considered doing part of that processing upfront, or organize your data differently so that handling at request-time becomes much simpler?
A lot of questions, and a lot of possibilities. It is hard to give one straight answer on such an open question. I hope I gave some food for thought nonetheless.
HTH!
answered Nov 22 at 19:30
grtjn
15.1k11730
These questions are tough to answer without properly knowing the whole situation. You talk about custom REST endpoints, but are using REST extensions on the built-in REST api for that purpose. Are you using anything else of the built-in REST api? You use search options, but don't use them for /v1/search. Could you have done the same with /v1/search with those options, but an extra REST Transform on top? And you speak of a lot of data mangling happening at sub-document level. Have you considered doing part of that processing upfront, or organize your data differently so that handling at request-time becomes much simpler?
A lot of questions, and a lot of possibilities. It is hard to give one straight answer on such an open question. I hope I gave some food for thought nonetheless.
HTH!
answered Nov 22 at 19:30
grtjn
15.1k11730
answered Nov 22 at 19:30
grtjn
15.1k11730
answered Nov 22 at 19:30
grtjn
15.1k11730
answered Nov 22 at 19:30
grtjn
15.1k11730
15.1k11730
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436604%2fmarklogic-design-question-on-query-options-vs-transform%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
