Deserialize hits memory limit
We have a custom apex webservice in which we do certain operations and later call a managed package apex method that returns a JSON string which we deserialize and send back to the client. For example purposes let's assume the following JSON structure is returned from the managed package method:
{
"x": "xxx",
"y": "yyy",
"z": "zzz",
"lines": [{
"a": "aaa",
"b": "bbb",
"c": "ccc",
"record": {
"attributes": {
"type": "Account",
"url": "/services/data/v44.0/sobjects/Account/001000000000000000"
},
"Id": "001000000000000000",
"AccountNumber": "12345",
"Contacts": {
"totalSize": 1,
"done": true,
"records": [{
"attributes": {
"type": "Contact",
"url": "/services/data/v44.0/sobjects/Contact/003S00000000000000"
},
"Id": "003S00000000000000",
"Phone": "22323423423",
"MobilePhone": "345345345345"
}]
}
}
}]
}
Taking the above example we could assume that we could deserialize the above JSON string into the following classes:
public class MainModel {
String x;
String y;
String z;
LinesModel lines;
}
public class LineModel {
String a;
String b;
String c;
Account record;
}
As we can see the LineModel
states that the record
attribute has an Account
sobject type, therefore it can span children relationships of Contacts, Opportunities and many other children relationships.
Therefore we could deserialize the JSON the following way:
MainModel model = (MainModel)JSON.deserialize(jsonString, MainModel.class);
Before returning the model to our webservice caller we want to get rid of some information that the manage package includes but we don't want, for example the Contacts
children tree for each Account
.
In order to remove it, we are doing the following:
Map<String, Object> genericModel = (Map<String, Object>)JSON.deserializeUntyped(jsonString);
Object lines = (Object)genericModel.get('lines');
Map<String, Object> itemObj;
Map<String, Object> record;
for (Object item : lines) {
itemObj = (Map<String, Object>)item;
record = (Map<String, Object>)itemObj.get('record');
if (record.containsKey('Contacts')) {
record.remove('Contacts')
}
}
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(genericModel), MainModel.class);
The issue we're facing is that when the response from the manage package has many nested records the desarialization process (the deserializeUntyped
call) fails with an Apex heap memory limit. When I say nested records, in the context of the above example there could maybe be only 20 Accounts but each account maybe having 60 contacts.
Since we don't have control on what the manage package returns in the JSON we cannot avoid having those unnecessary children.
We've tried removing the children using regex directly on the JSON string the following way to avoid unnecessary deserialization for cleaning:
String textPattern = '\"Contacts\":.*records.*\[.*]},';
String resultText = jsonString.replaceAll(textPattern, '');
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(resultText), MainModel.class);
But this results on a System.LimitException: Regex too complicated error because the JSON string is too large.
Any ideas on how we can clean the unwanted JSON data before returning it to the caller that could be viable?
Please take into account that the above is an example very simplified for the sake of understanding the real issue, the real JSON has many other relationships and objects that need cleaning and is more complex but the scenario is pretty much the same.
json deserialize heap
add a comment |
We have a custom apex webservice in which we do certain operations and later call a managed package apex method that returns a JSON string which we deserialize and send back to the client. For example purposes let's assume the following JSON structure is returned from the managed package method:
{
"x": "xxx",
"y": "yyy",
"z": "zzz",
"lines": [{
"a": "aaa",
"b": "bbb",
"c": "ccc",
"record": {
"attributes": {
"type": "Account",
"url": "/services/data/v44.0/sobjects/Account/001000000000000000"
},
"Id": "001000000000000000",
"AccountNumber": "12345",
"Contacts": {
"totalSize": 1,
"done": true,
"records": [{
"attributes": {
"type": "Contact",
"url": "/services/data/v44.0/sobjects/Contact/003S00000000000000"
},
"Id": "003S00000000000000",
"Phone": "22323423423",
"MobilePhone": "345345345345"
}]
}
}
}]
}
Taking the above example we could assume that we could deserialize the above JSON string into the following classes:
public class MainModel {
String x;
String y;
String z;
LinesModel lines;
}
public class LineModel {
String a;
String b;
String c;
Account record;
}
As we can see the LineModel
states that the record
attribute has an Account
sobject type, therefore it can span children relationships of Contacts, Opportunities and many other children relationships.
Therefore we could deserialize the JSON the following way:
MainModel model = (MainModel)JSON.deserialize(jsonString, MainModel.class);
Before returning the model to our webservice caller we want to get rid of some information that the manage package includes but we don't want, for example the Contacts
children tree for each Account
.
In order to remove it, we are doing the following:
Map<String, Object> genericModel = (Map<String, Object>)JSON.deserializeUntyped(jsonString);
Object lines = (Object)genericModel.get('lines');
Map<String, Object> itemObj;
Map<String, Object> record;
for (Object item : lines) {
itemObj = (Map<String, Object>)item;
record = (Map<String, Object>)itemObj.get('record');
if (record.containsKey('Contacts')) {
record.remove('Contacts')
}
}
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(genericModel), MainModel.class);
The issue we're facing is that when the response from the manage package has many nested records the desarialization process (the deserializeUntyped
call) fails with an Apex heap memory limit. When I say nested records, in the context of the above example there could maybe be only 20 Accounts but each account maybe having 60 contacts.
Since we don't have control on what the manage package returns in the JSON we cannot avoid having those unnecessary children.
We've tried removing the children using regex directly on the JSON string the following way to avoid unnecessary deserialization for cleaning:
String textPattern = '\"Contacts\":.*records.*\[.*]},';
String resultText = jsonString.replaceAll(textPattern, '');
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(resultText), MainModel.class);
But this results on a System.LimitException: Regex too complicated error because the JSON string is too large.
Any ideas on how we can clean the unwanted JSON data before returning it to the caller that could be viable?
Please take into account that the above is an example very simplified for the sake of understanding the real issue, the real JSON has many other relationships and objects that need cleaning and is more complex but the scenario is pretty much the same.
json deserialize heap
6
This doesn't sound like a memory leak to me, just a very large json string. The synchronous limit on heap size is ~6MB, and the async limit is ~12MB. If you're trying to manipulate that JSON, I'd expect that you'd only be able to handle a JSON string less than half that size (the string remains in memory, and you need at least as much memory to deserialize). Do you happen to have any idea of the size of your JSON string(s)?
– Derek F
2 days ago
1
I've edited the question title to reflect Derek's point that this isn't a leak but rather hitting the limit.
– Keith C
2 days ago
Great point @DerekF, indeed we'll need to take that into further consideration to avoid hitting the limits even if the deserialization doesn't include the non desired info
– jonathanwiesel
yesterday
Regex too complicated can usually be solved by breaking up a large string with substringBefore/substringAfter in a while loop. Collect the parts in a list and run the regex individually on them.
– Alexander Johannes
yesterday
add a comment |
We have a custom apex webservice in which we do certain operations and later call a managed package apex method that returns a JSON string which we deserialize and send back to the client. For example purposes let's assume the following JSON structure is returned from the managed package method:
{
"x": "xxx",
"y": "yyy",
"z": "zzz",
"lines": [{
"a": "aaa",
"b": "bbb",
"c": "ccc",
"record": {
"attributes": {
"type": "Account",
"url": "/services/data/v44.0/sobjects/Account/001000000000000000"
},
"Id": "001000000000000000",
"AccountNumber": "12345",
"Contacts": {
"totalSize": 1,
"done": true,
"records": [{
"attributes": {
"type": "Contact",
"url": "/services/data/v44.0/sobjects/Contact/003S00000000000000"
},
"Id": "003S00000000000000",
"Phone": "22323423423",
"MobilePhone": "345345345345"
}]
}
}
}]
}
Taking the above example we could assume that we could deserialize the above JSON string into the following classes:
public class MainModel {
String x;
String y;
String z;
LinesModel lines;
}
public class LineModel {
String a;
String b;
String c;
Account record;
}
As we can see the LineModel
states that the record
attribute has an Account
sobject type, therefore it can span children relationships of Contacts, Opportunities and many other children relationships.
Therefore we could deserialize the JSON the following way:
MainModel model = (MainModel)JSON.deserialize(jsonString, MainModel.class);
Before returning the model to our webservice caller we want to get rid of some information that the manage package includes but we don't want, for example the Contacts
children tree for each Account
.
In order to remove it, we are doing the following:
Map<String, Object> genericModel = (Map<String, Object>)JSON.deserializeUntyped(jsonString);
Object lines = (Object)genericModel.get('lines');
Map<String, Object> itemObj;
Map<String, Object> record;
for (Object item : lines) {
itemObj = (Map<String, Object>)item;
record = (Map<String, Object>)itemObj.get('record');
if (record.containsKey('Contacts')) {
record.remove('Contacts')
}
}
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(genericModel), MainModel.class);
The issue we're facing is that when the response from the manage package has many nested records the desarialization process (the deserializeUntyped
call) fails with an Apex heap memory limit. When I say nested records, in the context of the above example there could maybe be only 20 Accounts but each account maybe having 60 contacts.
Since we don't have control on what the manage package returns in the JSON we cannot avoid having those unnecessary children.
We've tried removing the children using regex directly on the JSON string the following way to avoid unnecessary deserialization for cleaning:
String textPattern = '\"Contacts\":.*records.*\[.*]},';
String resultText = jsonString.replaceAll(textPattern, '');
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(resultText), MainModel.class);
But this results on a System.LimitException: Regex too complicated error because the JSON string is too large.
Any ideas on how we can clean the unwanted JSON data before returning it to the caller that could be viable?
Please take into account that the above is an example very simplified for the sake of understanding the real issue, the real JSON has many other relationships and objects that need cleaning and is more complex but the scenario is pretty much the same.
json deserialize heap
We have a custom apex webservice in which we do certain operations and later call a managed package apex method that returns a JSON string which we deserialize and send back to the client. For example purposes let's assume the following JSON structure is returned from the managed package method:
{
"x": "xxx",
"y": "yyy",
"z": "zzz",
"lines": [{
"a": "aaa",
"b": "bbb",
"c": "ccc",
"record": {
"attributes": {
"type": "Account",
"url": "/services/data/v44.0/sobjects/Account/001000000000000000"
},
"Id": "001000000000000000",
"AccountNumber": "12345",
"Contacts": {
"totalSize": 1,
"done": true,
"records": [{
"attributes": {
"type": "Contact",
"url": "/services/data/v44.0/sobjects/Contact/003S00000000000000"
},
"Id": "003S00000000000000",
"Phone": "22323423423",
"MobilePhone": "345345345345"
}]
}
}
}]
}
Taking the above example we could assume that we could deserialize the above JSON string into the following classes:
public class MainModel {
String x;
String y;
String z;
LinesModel lines;
}
public class LineModel {
String a;
String b;
String c;
Account record;
}
As we can see the LineModel
states that the record
attribute has an Account
sobject type, therefore it can span children relationships of Contacts, Opportunities and many other children relationships.
Therefore we could deserialize the JSON the following way:
MainModel model = (MainModel)JSON.deserialize(jsonString, MainModel.class);
Before returning the model to our webservice caller we want to get rid of some information that the manage package includes but we don't want, for example the Contacts
children tree for each Account
.
In order to remove it, we are doing the following:
Map<String, Object> genericModel = (Map<String, Object>)JSON.deserializeUntyped(jsonString);
Object lines = (Object)genericModel.get('lines');
Map<String, Object> itemObj;
Map<String, Object> record;
for (Object item : lines) {
itemObj = (Map<String, Object>)item;
record = (Map<String, Object>)itemObj.get('record');
if (record.containsKey('Contacts')) {
record.remove('Contacts')
}
}
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(genericModel), MainModel.class);
The issue we're facing is that when the response from the manage package has many nested records the desarialization process (the deserializeUntyped
call) fails with an Apex heap memory limit. When I say nested records, in the context of the above example there could maybe be only 20 Accounts but each account maybe having 60 contacts.
Since we don't have control on what the manage package returns in the JSON we cannot avoid having those unnecessary children.
We've tried removing the children using regex directly on the JSON string the following way to avoid unnecessary deserialization for cleaning:
String textPattern = '\"Contacts\":.*records.*\[.*]},';
String resultText = jsonString.replaceAll(textPattern, '');
MainModel model = (MainModel)JSON.deserialize(JSON.serialize(resultText), MainModel.class);
But this results on a System.LimitException: Regex too complicated error because the JSON string is too large.
Any ideas on how we can clean the unwanted JSON data before returning it to the caller that could be viable?
Please take into account that the above is an example very simplified for the sake of understanding the real issue, the real JSON has many other relationships and objects that need cleaning and is more complex but the scenario is pretty much the same.
json deserialize heap
json deserialize heap
edited 2 days ago
Keith C
94.7k1089204
94.7k1089204
asked 2 days ago
jonathanwieseljonathanwiesel
257114
257114
6
This doesn't sound like a memory leak to me, just a very large json string. The synchronous limit on heap size is ~6MB, and the async limit is ~12MB. If you're trying to manipulate that JSON, I'd expect that you'd only be able to handle a JSON string less than half that size (the string remains in memory, and you need at least as much memory to deserialize). Do you happen to have any idea of the size of your JSON string(s)?
– Derek F
2 days ago
1
I've edited the question title to reflect Derek's point that this isn't a leak but rather hitting the limit.
– Keith C
2 days ago
Great point @DerekF, indeed we'll need to take that into further consideration to avoid hitting the limits even if the deserialization doesn't include the non desired info
– jonathanwiesel
yesterday
Regex too complicated can usually be solved by breaking up a large string with substringBefore/substringAfter in a while loop. Collect the parts in a list and run the regex individually on them.
– Alexander Johannes
yesterday
add a comment |
6
This doesn't sound like a memory leak to me, just a very large json string. The synchronous limit on heap size is ~6MB, and the async limit is ~12MB. If you're trying to manipulate that JSON, I'd expect that you'd only be able to handle a JSON string less than half that size (the string remains in memory, and you need at least as much memory to deserialize). Do you happen to have any idea of the size of your JSON string(s)?
– Derek F
2 days ago
1
I've edited the question title to reflect Derek's point that this isn't a leak but rather hitting the limit.
– Keith C
2 days ago
Great point @DerekF, indeed we'll need to take that into further consideration to avoid hitting the limits even if the deserialization doesn't include the non desired info
– jonathanwiesel
yesterday
Regex too complicated can usually be solved by breaking up a large string with substringBefore/substringAfter in a while loop. Collect the parts in a list and run the regex individually on them.
– Alexander Johannes
yesterday
6
6
This doesn't sound like a memory leak to me, just a very large json string. The synchronous limit on heap size is ~6MB, and the async limit is ~12MB. If you're trying to manipulate that JSON, I'd expect that you'd only be able to handle a JSON string less than half that size (the string remains in memory, and you need at least as much memory to deserialize). Do you happen to have any idea of the size of your JSON string(s)?
– Derek F
2 days ago
This doesn't sound like a memory leak to me, just a very large json string. The synchronous limit on heap size is ~6MB, and the async limit is ~12MB. If you're trying to manipulate that JSON, I'd expect that you'd only be able to handle a JSON string less than half that size (the string remains in memory, and you need at least as much memory to deserialize). Do you happen to have any idea of the size of your JSON string(s)?
– Derek F
2 days ago
1
1
I've edited the question title to reflect Derek's point that this isn't a leak but rather hitting the limit.
– Keith C
2 days ago
I've edited the question title to reflect Derek's point that this isn't a leak but rather hitting the limit.
– Keith C
2 days ago
Great point @DerekF, indeed we'll need to take that into further consideration to avoid hitting the limits even if the deserialization doesn't include the non desired info
– jonathanwiesel
yesterday
Great point @DerekF, indeed we'll need to take that into further consideration to avoid hitting the limits even if the deserialization doesn't include the non desired info
– jonathanwiesel
yesterday
Regex too complicated can usually be solved by breaking up a large string with substringBefore/substringAfter in a while loop. Collect the parts in a list and run the regex individually on them.
– Alexander Johannes
yesterday
Regex too complicated can usually be solved by breaking up a large string with substringBefore/substringAfter in a while loop. Collect the parts in a list and run the regex individually on them.
– Alexander Johannes
yesterday
add a comment |
1 Answer
1
active
oldest
votes
In your wrapper class LineModel
, don't use Standard Account. If you do that you will also deserialize children. Instead, create Apex Wrapper for Account with fields you need. This will be more memory efficient.
public class LineModel {
String a;
String b;
String c;
MyAccount record;
}
public Class MyAccount{
String Id;
String AccountNumber;
}
public class MainModel {
String x;
String y;
String z;
List<LineModel> lines;
}
1
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. theattributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work withJSON.deserializeStrict()
. +1
– Derek F
2 days ago
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause anInvalid character in identifier: customfield__c
error.
– jonathanwiesel
yesterday
I'm aware that using different names in the class (jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again theRegex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string
– jonathanwiesel
yesterday
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "459"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f246040%2fdeserialize-hits-memory-limit%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In your wrapper class LineModel
, don't use Standard Account. If you do that you will also deserialize children. Instead, create Apex Wrapper for Account with fields you need. This will be more memory efficient.
public class LineModel {
String a;
String b;
String c;
MyAccount record;
}
public Class MyAccount{
String Id;
String AccountNumber;
}
public class MainModel {
String x;
String y;
String z;
List<LineModel> lines;
}
1
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. theattributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work withJSON.deserializeStrict()
. +1
– Derek F
2 days ago
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause anInvalid character in identifier: customfield__c
error.
– jonathanwiesel
yesterday
I'm aware that using different names in the class (jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again theRegex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string
– jonathanwiesel
yesterday
add a comment |
In your wrapper class LineModel
, don't use Standard Account. If you do that you will also deserialize children. Instead, create Apex Wrapper for Account with fields you need. This will be more memory efficient.
public class LineModel {
String a;
String b;
String c;
MyAccount record;
}
public Class MyAccount{
String Id;
String AccountNumber;
}
public class MainModel {
String x;
String y;
String z;
List<LineModel> lines;
}
1
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. theattributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work withJSON.deserializeStrict()
. +1
– Derek F
2 days ago
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause anInvalid character in identifier: customfield__c
error.
– jonathanwiesel
yesterday
I'm aware that using different names in the class (jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again theRegex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string
– jonathanwiesel
yesterday
add a comment |
In your wrapper class LineModel
, don't use Standard Account. If you do that you will also deserialize children. Instead, create Apex Wrapper for Account with fields you need. This will be more memory efficient.
public class LineModel {
String a;
String b;
String c;
MyAccount record;
}
public Class MyAccount{
String Id;
String AccountNumber;
}
public class MainModel {
String x;
String y;
String z;
List<LineModel> lines;
}
In your wrapper class LineModel
, don't use Standard Account. If you do that you will also deserialize children. Instead, create Apex Wrapper for Account with fields you need. This will be more memory efficient.
public class LineModel {
String a;
String b;
String c;
MyAccount record;
}
public Class MyAccount{
String Id;
String AccountNumber;
}
public class MainModel {
String x;
String y;
String z;
List<LineModel> lines;
}
edited 2 days ago
Jayant Das
12.8k2723
12.8k2723
answered 2 days ago
Pranay JaiswalPranay Jaiswal
14.2k32452
14.2k32452
1
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. theattributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work withJSON.deserializeStrict()
. +1
– Derek F
2 days ago
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause anInvalid character in identifier: customfield__c
error.
– jonathanwiesel
yesterday
I'm aware that using different names in the class (jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again theRegex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string
– jonathanwiesel
yesterday
add a comment |
1
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. theattributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work withJSON.deserializeStrict()
. +1
– Derek F
2 days ago
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause anInvalid character in identifier: customfield__c
error.
– jonathanwiesel
yesterday
I'm aware that using different names in the class (jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again theRegex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string
– jonathanwiesel
yesterday
1
1
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
+1. This is indeed a very neat approach to deal with any such limitations.
– Jayant Das
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. the
attributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work with JSON.deserializeStrict()
. +1– Derek F
2 days ago
Somehow, I didn't expect this to work (given that serialized SObjects contain extra data, i.e. the
attributes
attribute). However, copy/pasting the JSON from the question, and your deserialization classes, it all worked without a hitch. After mulling it over a bit, I suppose I shouldn't have been surprised. The only gotcha here is that this won't work with JSON.deserializeStrict()
. +1– Derek F
2 days ago
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause an
Invalid character in identifier: customfield__c
error.– jonathanwiesel
yesterday
This seems like a great solution, in fact I marked as the correct answer until I figured out that if custom fields are included in the JSON they cannot be added to the serialization class because an attribute name with double underscores will cause an
Invalid character in identifier: customfield__c
error.– jonathanwiesel
yesterday
I'm aware that using different names in the class (
jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again the Regex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string– jonathanwiesel
yesterday
I'm aware that using different names in the class (
jsonString.replaceAll('customfield__c', 'customfield_X)
) could work but could that could hit again the Regex too complicated error
? Maybe I'm wrong since it's not a regex but a plain string, yet I'm concerned on the CPU limit while replacing 50 custom fields in a 2MB string– jonathanwiesel
yesterday
add a comment |
Thanks for contributing an answer to Salesforce Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f246040%2fdeserialize-hits-memory-limit%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
6
This doesn't sound like a memory leak to me, just a very large json string. The synchronous limit on heap size is ~6MB, and the async limit is ~12MB. If you're trying to manipulate that JSON, I'd expect that you'd only be able to handle a JSON string less than half that size (the string remains in memory, and you need at least as much memory to deserialize). Do you happen to have any idea of the size of your JSON string(s)?
– Derek F
2 days ago
1
I've edited the question title to reflect Derek's point that this isn't a leak but rather hitting the limit.
– Keith C
2 days ago
Great point @DerekF, indeed we'll need to take that into further consideration to avoid hitting the limits even if the deserialization doesn't include the non desired info
– jonathanwiesel
yesterday
Regex too complicated can usually be solved by breaking up a large string with substringBefore/substringAfter in a while loop. Collect the parts in a list and run the regex individually on them.
– Alexander Johannes
yesterday