pandas to_csv() fails to save the results
I am using Pandas for creating a data frame that reads values from a comma separated file initially. The CSV file contains a dataset related to the employees. The data from the csv file is normalized between the range of 0-1 using MinMaxScaler. The normalization step works fine and the print statement prints out the normalizaed results of the attributes. But when I try to save these results (normalized values) against each attribute in a new CSV file, it creates a new file "Employees_modified" with the same values as that of the "Employees" dataset that were given as input to the MinMaxScaler function. I am new to Pandas and couldn't figure out where possibly is the mistake.
My code is given below:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
dframe.to_csv('Employees_modified.csv')
I want to save normalized values in my Employees_modified file but the results are being lost somewhere in between.
python pandas csv
edited Nov 24 at 5:15
jwodder
32.8k35081
asked Nov 22 at 19:38
user399
45
add a comment |
I am using Pandas for creating a data frame that reads values from a comma separated file initially. The CSV file contains a dataset related to the employees. The data from the csv file is normalized between the range of 0-1 using MinMaxScaler. The normalization step works fine and the print statement prints out the normalizaed results of the attributes. But when I try to save these results (normalized values) against each attribute in a new CSV file, it creates a new file "Employees_modified" with the same values as that of the "Employees" dataset that were given as input to the MinMaxScaler function. I am new to Pandas and couldn't figure out where possibly is the mistake.
My code is given below:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
dframe.to_csv('Employees_modified.csv')
I want to save normalized values in my Employees_modified file but the results are being lost somewhere in between.
python pandas csv
edited Nov 24 at 5:15
jwodder
32.8k35081
asked Nov 22 at 19:38
user399
45
1
What do you mean it's lost? Do you get any output file at all? What's in it and what do you expect to be in it?
– kabdulla
Nov 22 at 19:47
2
you're saving the original dataframe, not the result of the transformation...
– godot
Nov 22 at 19:52
add a comment |
I am using Pandas for creating a data frame that reads values from a comma separated file initially. The CSV file contains a dataset related to the employees. The data from the csv file is normalized between the range of 0-1 using MinMaxScaler. The normalization step works fine and the print statement prints out the normalizaed results of the attributes. But when I try to save these results (normalized values) against each attribute in a new CSV file, it creates a new file "Employees_modified" with the same values as that of the "Employees" dataset that were given as input to the MinMaxScaler function. I am new to Pandas and couldn't figure out where possibly is the mistake.
My code is given below:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
dframe.to_csv('Employees_modified.csv')
I want to save normalized values in my Employees_modified file but the results are being lost somewhere in between.
python pandas csv
edited Nov 24 at 5:15
jwodder
32.8k35081
asked Nov 22 at 19:38
user399
45
I am using Pandas for creating a data frame that reads values from a comma separated file initially. The CSV file contains a dataset related to the employees. The data from the csv file is normalized between the range of 0-1 using MinMaxScaler. The normalization step works fine and the print statement prints out the normalizaed results of the attributes. But when I try to save these results (normalized values) against each attribute in a new CSV file, it creates a new file "Employees_modified" with the same values as that of the "Employees" dataset that were given as input to the MinMaxScaler function. I am new to Pandas and couldn't figure out where possibly is the mistake.
My code is given below:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
dframe.to_csv('Employees_modified.csv')
I want to save normalized values in my Employees_modified file but the results are being lost somewhere in between.
python pandas csv
python pandas csv
edited Nov 24 at 5:15
jwodder
32.8k35081
asked Nov 22 at 19:38
user399
45
edited Nov 24 at 5:15
jwodder
32.8k35081
asked Nov 22 at 19:38
user399
45
edited Nov 24 at 5:15
jwodder
32.8k35081
edited Nov 24 at 5:15
jwodder
32.8k35081
edited Nov 24 at 5:15
jwodder
32.8k35081
32.8k35081
asked Nov 22 at 19:38
user399
45
asked Nov 22 at 19:38
user399
45
asked Nov 22 at 19:38
user399
45
45
1
What do you mean it's lost? Do you get any output file at all? What's in it and what do you expect to be in it?
– kabdulla
Nov 22 at 19:47
2
you're saving the original dataframe, not the result of the transformation...
– godot
Nov 22 at 19:52
add a comment |
1
What do you mean it's lost? Do you get any output file at all? What's in it and what do you expect to be in it?
– kabdulla
Nov 22 at 19:47
2
you're saving the original dataframe, not the result of the transformation...
– godot
Nov 22 at 19:52
1
1
What do you mean it's lost? Do you get any output file at all? What's in it and what do you expect to be in it?
– kabdulla
Nov 22 at 19:47
What do you mean it's lost? Do you get any output file at all? What's in it and what do you expect to be in it?
– kabdulla
Nov 22 at 19:47
2
2
you're saving the original dataframe, not the result of the transformation...
– godot
Nov 22 at 19:52
you're saving the original dataframe, not the result of the transformation...
– godot
Nov 22 at 19:52
add a comment |
2 Answers
2
active
oldest
votes
You are writing the original dataframe to the file.
The code below should do what you want:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
#creates a dataframe from the scaled data
pd.DataFrame(a_scaled, columns=list(dframe)).to_csv('Employees_modified.csv')
answered Nov 22 at 19:55
Pedro Torres
683413
add a comment |
Try:
one_scaler = MinMaxScaler(feature_range=(0, 1), copy=False)
answered Nov 22 at 19:53
AResem
1114
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53437191%2fpandas-to-csv-fails-to-save-the-results%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You are writing the original dataframe to the file.
The code below should do what you want:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
#creates a dataframe from the scaled data
pd.DataFrame(a_scaled, columns=list(dframe)).to_csv('Employees_modified.csv')
answered Nov 22 at 19:55
Pedro Torres
683413
add a comment |
You are writing the original dataframe to the file.
The code below should do what you want:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
#creates a dataframe from the scaled data
pd.DataFrame(a_scaled, columns=list(dframe)).to_csv('Employees_modified.csv')
answered Nov 22 at 19:55
Pedro Torres
683413
add a comment |
You are writing the original dataframe to the file.
The code below should do what you want:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
#creates a dataframe from the scaled data
pd.DataFrame(a_scaled, columns=list(dframe)).to_csv('Employees_modified.csv')
answered Nov 22 at 19:55
Pedro Torres
683413
You are writing the original dataframe to the file.
The code below should do what you want:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
dframe = pd.read_csv('Employees.csv')
one_scaler = MinMaxScaler(feature_range=(0, 1))
one_scaler.partial_fit(dframe)
a_scaled = one_scaler.transform(dframe)
print(a_scaled)
#creates a dataframe from the scaled data
pd.DataFrame(a_scaled, columns=list(dframe)).to_csv('Employees_modified.csv')
answered Nov 22 at 19:55
Pedro Torres
683413
answered Nov 22 at 19:55
Pedro Torres
683413
answered Nov 22 at 19:55
Pedro Torres
683413
answered Nov 22 at 19:55
Pedro Torres
683413
683413
add a comment |
add a comment |
Try:
one_scaler = MinMaxScaler(feature_range=(0, 1), copy=False)
answered Nov 22 at 19:53
AResem
1114
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
add a comment |
Try:
one_scaler = MinMaxScaler(feature_range=(0, 1), copy=False)
answered Nov 22 at 19:53
AResem
1114
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
add a comment |
Try:
one_scaler = MinMaxScaler(feature_range=(0, 1), copy=False)
answered Nov 22 at 19:53
AResem
1114
Try:
one_scaler = MinMaxScaler(feature_range=(0, 1), copy=False)
answered Nov 22 at 19:53
AResem
1114
answered Nov 22 at 19:53
AResem
1114
answered Nov 22 at 19:53
AResem
1114
answered Nov 22 at 19:53
AResem
1114
1114
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
add a comment |
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
thank you for providing an answer. Please provide more information as to how the changes you've proposed in your answer will help to fix the issue in question.
– Bryce Siedschlaw
Nov 22 at 21:29
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
Try to run the same code just changing the line I pointed out. By default, MinMaxScaler operates on a copy of your dataFrame. So when you print your original dataframe nothing changed. The argument 'copy=False' tells the scaler to not copy, but operate over your original dataframe. I hope it helps.
– AResem
Nov 22 at 23:06
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
@AResem I appreciate your contribution but your proposed solution didn't make any change to my Employee_modified file
– user399
Nov 23 at 0:55
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53437191%2fpandas-to-csv-fails-to-save-the-results%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
What do you mean it's lost? Do you get any output file at all? What's in it and what do you expect to be in it?
– kabdulla
Nov 22 at 19:47
2
you're saving the original dataframe, not the result of the transformation...
– godot
Nov 22 at 19:52