I think we should ignore channels for this project, the data is not that important so any files named channel.txt can be ignored. What I need: Folders/Files in this structure: channelid/year/month/userid.txt Example: 11148817/2018/12/22484632.txt Current data is similar, but uses usernames and month is a string instead of int. ALSO Files can be .txt or txt.gz in the old logs, new logs should only be .txt I will gzip them later myself. Current lines are saved like this: [2017-12-31 23:15:32] gempir: FeelsWeirdMan This the new format: @badges=subscriber/6,premium/1;color=#FF0000;display-name=Redflamingo13;emotes=;id=2a31a9df-d6ff-4840-b211-a2547c7e656e;mod=0;room-id=11148817;subscriber=1;tmi-sent-ts=1490382457309;turbo=0;user-id=78424343;user-type= :[email protected] PRIVMSG #pajlada :Thrashh5, FeelsWayTooAmazingMan kinda For more explanations of these parts this test might be helpful: https://github.com/gempir/go-twitch-irc/blob/master/message_test.go Obviously we can't get all that data from just the line of txt. but what we should be able to get is: room-id, tmi-sent-ts (this is very important for the timestamp in the end), user-id, and maybe display-name if you feel like it. Almost all messages are PRIVMSG so you don't need to do different types just PRIVMSG.
Comments