How to read tweet/text with ellipsis in R
I am trying to read tweets from a bunch of different text files and then count the number of characters in each tweet.
The code I am using the read the individual text files is the following:
nbc <- readLines(".../nbchealthnews.txt",
encoding = "utf-10") %>%
map(., str_split_fixed, "\|", 3) %>%
map_df(., as_tibble)
I then want to run
nbc_tweetLength <- nchar(nbc$V3)
However, I get the following error:
> nbc_tweetLength <- nchar(nbc$V3)
Error in nchar(nbc$V3) : invalid multibyte string, element 271
Element 271 is
> nbc$V3[271]
[1] "RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: u0085"
while the corresponding actual tweet in the text file is
RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: …
How I can read the tweet as it is. That is, read the ellipsis that appears after the colon as it is, so that the text of the tweet remains unchanged?
If that is not possible, how can I circumvent the issue of counting the total number of characters in each tweet while accounting for special characters such as x85 and u0092 (the latter appears in another tweet when it is read into R; in the original text, this is a curly apostrophe (’).
r
asked Nov 22 at 18:32
Anonymouse
527
add a comment |
I am trying to read tweets from a bunch of different text files and then count the number of characters in each tweet.
The code I am using the read the individual text files is the following:
nbc <- readLines(".../nbchealthnews.txt",
encoding = "utf-10") %>%
map(., str_split_fixed, "\|", 3) %>%
map_df(., as_tibble)
I then want to run
nbc_tweetLength <- nchar(nbc$V3)
However, I get the following error:
> nbc_tweetLength <- nchar(nbc$V3)
Error in nchar(nbc$V3) : invalid multibyte string, element 271
Element 271 is
> nbc$V3[271]
[1] "RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: u0085"
while the corresponding actual tweet in the text file is
RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: …
How I can read the tweet as it is. That is, read the ellipsis that appears after the colon as it is, so that the text of the tweet remains unchanged?
If that is not possible, how can I circumvent the issue of counting the total number of characters in each tweet while accounting for special characters such as x85 and u0092 (the latter appears in another tweet when it is read into R; in the original text, this is a curly apostrophe (’).
r
asked Nov 22 at 18:32
Anonymouse
527
1
Trystringi::stri_read_linesinstead ofreadLinesandstringi::stri_lengthand read up instringi's docs on how to convert to UTF-8
– hrbrmstr
Nov 22 at 18:39
@hrbrmstr : Thanks for your help! Usingstringi::stri_read_lineswith"encoding = auto"seems to do the trick!
– Anonymouse
Nov 22 at 19:10
add a comment |
I am trying to read tweets from a bunch of different text files and then count the number of characters in each tweet.
The code I am using the read the individual text files is the following:
nbc <- readLines(".../nbchealthnews.txt",
encoding = "utf-10") %>%
map(., str_split_fixed, "\|", 3) %>%
map_df(., as_tibble)
I then want to run
nbc_tweetLength <- nchar(nbc$V3)
However, I get the following error:
> nbc_tweetLength <- nchar(nbc$V3)
Error in nchar(nbc$V3) : invalid multibyte string, element 271
Element 271 is
> nbc$V3[271]
[1] "RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: u0085"
while the corresponding actual tweet in the text file is
RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: …
How I can read the tweet as it is. That is, read the ellipsis that appears after the colon as it is, so that the text of the tweet remains unchanged?
If that is not possible, how can I circumvent the issue of counting the total number of characters in each tweet while accounting for special characters such as x85 and u0092 (the latter appears in another tweet when it is read into R; in the original text, this is a curly apostrophe (’).
r
asked Nov 22 at 18:32
Anonymouse
527
I am trying to read tweets from a bunch of different text files and then count the number of characters in each tweet.
The code I am using the read the individual text files is the following:
nbc <- readLines(".../nbchealthnews.txt",
encoding = "utf-10") %>%
map(., str_split_fixed, "\|", 3) %>%
map_df(., as_tibble)
I then want to run
nbc_tweetLength <- nchar(nbc$V3)
However, I get the following error:
> nbc_tweetLength <- nchar(nbc$V3)
Error in nchar(nbc$V3) : invalid multibyte string, element 271
Element 271 is
> nbc$V3[271]
[1] "RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: u0085"
while the corresponding actual tweet in the text file is
RT @JuliaSommerfeld: Tales of chucking big jobs are the new lady porn RT @ELLEmagazine: What's REALLY causing women to burn out before 30: …
How I can read the tweet as it is. That is, read the ellipsis that appears after the colon as it is, so that the text of the tweet remains unchanged?
If that is not possible, how can I circumvent the issue of counting the total number of characters in each tweet while accounting for special characters such as x85 and u0092 (the latter appears in another tweet when it is read into R; in the original text, this is a curly apostrophe (’).
r
r
asked Nov 22 at 18:32
Anonymouse
527
asked Nov 22 at 18:32
Anonymouse
527
edited Nov 22 at 18:41
asked Nov 22 at 18:32
Anonymouse
527
asked Nov 22 at 18:32
Anonymouse
527
asked Nov 22 at 18:32
Anonymouse
527
527
1
Trystringi::stri_read_linesinstead ofreadLinesandstringi::stri_lengthand read up instringi's docs on how to convert to UTF-8
– hrbrmstr
Nov 22 at 18:39
@hrbrmstr : Thanks for your help! Usingstringi::stri_read_lineswith"encoding = auto"seems to do the trick!
– Anonymouse
Nov 22 at 19:10
add a comment |
1
Trystringi::stri_read_linesinstead ofreadLinesandstringi::stri_lengthand read up instringi's docs on how to convert to UTF-8
– hrbrmstr
Nov 22 at 18:39
@hrbrmstr : Thanks for your help! Usingstringi::stri_read_lineswith"encoding = auto"seems to do the trick!
– Anonymouse
Nov 22 at 19:10
1
1
Try
stringi::stri_read_lines instead of readLines and stringi::stri_length and read up in stringi's docs on how to convert to UTF-8– hrbrmstr
Nov 22 at 18:39
Try
stringi::stri_read_lines instead of readLines and stringi::stri_length and read up in stringi's docs on how to convert to UTF-8– hrbrmstr
Nov 22 at 18:39
@hrbrmstr : Thanks for your help! Using
stringi::stri_read_lines with "encoding = auto" seems to do the trick!– Anonymouse
Nov 22 at 19:10
@hrbrmstr : Thanks for your help! Using
stringi::stri_read_lines with "encoding = auto" seems to do the trick!– Anonymouse
Nov 22 at 19:10
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436511%2fhow-to-read-tweet-text-with-ellipsis-in-r%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436511%2fhow-to-read-tweet-text-with-ellipsis-in-r%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Try
stringi::stri_read_linesinstead ofreadLinesandstringi::stri_lengthand read up instringi's docs on how to convert to UTF-8– hrbrmstr
Nov 22 at 18:39
@hrbrmstr : Thanks for your help! Using
stringi::stri_read_lineswith"encoding = auto"seems to do the trick!– Anonymouse
Nov 22 at 19:10