Tagged: summary analysis large datasets
February 9, 2016 at 8:52 pm #655
i am collecting tweeets for some period of time for the purpose of our research. As there is limit per google sheet – we have to make different sheet for every 20 key words (accounts we follow) – but now would like to do analysis of merged data sets using summary sheet (more then lets say 5 sheets with 20 accounts merged together) I tried to merge 20 accounts + 20 accounts – delete summary sheet, and create new one – and its working – but not sure for how many rows in total will work… any experience with similar situations?
Or is it possible to download sheets and then merge them in excel on the laptop and then to summary analysis
SergejFebruary 10, 2016 at 4:44 am #656
and paper we published is here
https://www.researchgate.net/publication/292392240_An_Analysis_of_Twitter_Usage_Among_Startups_in_EuropeFebruary 11, 2016 at 6:48 pm #657
Merging offline makes sense, Google Sheets has a 2m cell limit per spreadsheet. You might want to look at OpenRefine as a tool, it can import and merge multiple .CSV files. Another tool I’ve seen used with TAGS is R – this however needs you to write scriptsFebruary 13, 2016 at 12:41 pm #658
Dear Martin, thank you for feedback. Checked OpenRefine, but as its working as a part of google spreadsheets (if I understand it correctly) then with my data set will hardly work (as its too big), so another direction should be R. Actually I was thinking about it as a option, but I never worked with it. So will need to find a help with it.
Feel free to post link to paper we published about our research. There are some quite interesting finding so far.
SergejFebruary 13, 2016 at 3:31 pm #659
OpenRefine can integrate with Google Sheets but also works on standalone files. It runs off your desktop and can easily handle millions of rows. Workflow that might work for you is download individual TAGS archives, merge in OpenRefine then export a single file for something else … Excel, Tableau etc. (you could merge the data in Excel but personally I’d use OpenRefine to do this)February 14, 2016 at 9:42 pm #660
Thank you Martin, so actually use OpenRefine to merge all data and then use some other SW to cary on analysis
will try to do it… thank you for support
You must be logged in to reply to this topic.