Latest Diablo 3 News
DiabloWiki Updates
Support the site! Become a Diablo: IncGamers PAL - Remove ads and more!

CSV question

Discussion in 'Off-Topic' started by Dondrei, May 13, 2008.

  1. Dondrei

    Dondrei IncGamers Member

    Joined:
    Aug 1, 2005
    Messages:
    36,855
    Likes Received:
    30
    Trophy Points:
    467
    CSV question

    At work we regularly work with large datasets which we sometimes have to export in CSV format for one reason or another. I'd like to be able to run a simple check of the number of records (ie rows, not entries) to make sure it exported correctly, now if the file is small enough I can open it in Excel and check that way but frequently it isn't.

    My crappy workaround is to open it in Word and use word count to get the number of lines (I told you it was crappy), but I'd love to know if there was a better way. I mean, for one thing I have to set the page margins to zero and put it landscape to make sure all rows only take up one line... plus I have to wait forever while Word calculates the word count... it completely sucks.

    Does anyone know of a better way? I can't really use any utilities I need to download since this is a work computer and clearing an app with IT is like pulling teeth.
     
  2. garublador

    garublador IncGamers Member

    Joined:
    May 28, 2004
    Messages:
    1,326
    Likes Received:
    0
    Trophy Points:
    105
    Re: CSV question

    You can use the "find" command in DOS with the "/c" option and look for a line feed. I'm not sure how to get the line feed character right in DOS, but I'm sure there's a way.
     
  3. Eritar

    Eritar IncGamers Member

    Joined:
    Jun 22, 2003
    Messages:
    58
    Likes Received:
    0
    Trophy Points:
    11
    Re: CSV question

    Pop this in a notepad file, save it as a .vbs, and then run it.

    Code:
    Const ForReading = 1
    
    Dim filePath
    
    filePath = Inputbox("Enter the file path", "Amazing Linecount Utility©")
    
    Set objFSO = CreateObject("Scripting.FileSystemObject")
    Set objTextFile = objFSO.OpenTextFile(filePath, ForReading)
    
    objTextFile.ReadAll
    Wscript.Echo "Number of lines: " & objTextFile.Line
    
    
    Source
     
  4. krischan

    krischan Europe Trade Moderator

    Joined:
    Aug 17, 2003
    Messages:
    29,907
    Likes Received:
    914
    Trophy Points:
    416
    Re: CSV question

    Any text editor worth mentioning displays the number of lines somewhere.

    Unix/Linux: wc -l file.csv :tongue:
     
  5. Dondrei

    Dondrei IncGamers Member

    Joined:
    Aug 1, 2005
    Messages:
    36,855
    Likes Received:
    30
    Trophy Points:
    467
    Re: CSV question

    Thanks, I'll give those things a shot when I'm at work.

    Notepad/Wordpad aren't worth mentioning.



     
  6. garublador

    garublador IncGamers Member

    Joined:
    May 28, 2004
    Messages:
    1,326
    Likes Received:
    0
    Trophy Points:
    105
    Re: CSV question

    With mine, you can also use the /v modifier (display lines without the given search string) if you can't get it to work with searching for line feeds. You just have to search for a string that you know doesn't exist in any of the lines.

    The other method looks pretty easy, though.



     
  7. HolyOne

    HolyOne IncGamers Member

    Joined:
    Aug 22, 2006
    Messages:
    22
    Likes Received:
    0
    Trophy Points:
    5
    Re: CSV question

    Notepad++ is free and it says lines.
    you got loads of options here :) pick 1
     
  8. Dondrei

    Dondrei IncGamers Member

    Joined:
    Aug 1, 2005
    Messages:
    36,855
    Likes Received:
    30
    Trophy Points:
    467
    Re: CSV question

    Hey thanks Eritar, that vbscript worked like a charm.

    Yeah, I know there are free utilities but getting anything past IT security is a pain in the backside. In fact they're even less likely to let it through if it's freeware.



     
  9. 5Ws

    5Ws Banned

    Joined:
    Dec 2, 2004
    Messages:
    1,706
    Likes Received:
    0
    Trophy Points:
    0
    Re: CSV question

    What environment are you working in? If it's Linux, just use vi or emacs.
    How large is your large dataset? I frequently work with sets that have millions of observations and upwards of 500 variables, and I can tell you, number of rows is the least of your worries in exporting/importing.
    What format is the original dataset in? And what software do you use to do the CSV conversion?



     
  10. krischan

    krischan Europe Trade Moderator

    Joined:
    Aug 17, 2003
    Messages:
    29,907
    Likes Received:
    914
    Trophy Points:
    416
    Re: CSV question

    . .



     
  11. 5Ws

    5Ws Banned

    Joined:
    Dec 2, 2004
    Messages:
    1,706
    Likes Received:
    0
    Trophy Points:
    0
    Re: CSV question

    Great, learned a new command today! Heh, I always open the whole thing with vi and ctrl+g to the bottom.



     
  12. krischan

    krischan Europe Trade Moderator

    Joined:
    Aug 17, 2003
    Messages:
    29,907
    Likes Received:
    914
    Trophy Points:
    416
    Re: CSV question

    Nitpicking: Ctrl-G shows the number of lines without going to the bottom, but that will do the job even better :azn:
     
  13. Dondrei

    Dondrei IncGamers Member

    Joined:
    Aug 1, 2005
    Messages:
    36,855
    Likes Received:
    30
    Trophy Points:
    467
    Re: CSV question

    The situation is kind of complicated, basically most things are on Win 2000 but we use SAS EG on a dedicated SAS server, which is Unix. I have Putty/telnet but my shell access is restricted (I should really talk to the guys in Infrastructure at some point) so basically the most powerful command at my disposal is ls. And I can't access any directories through that.

    We have several products with over a million accounts each, never mind daily transactions for each account and so on, and we have a ****load of tables with ~500 variables (and yet they give me a 14Gb server quota, LO-****ing-L). But since all the load happens on the server rather than on our (ancient) local machines size really isn't the problem, by far our biggest problem is the interface between Unix and Windows environments. More specifically lack thereof (there are actually plenty of ways to fix that but the business DELIBERATELY does not implement them for a variety of irritating reasons).



     
  14. 5Ws

    5Ws Banned

    Joined:
    Dec 2, 2004
    Messages:
    1,706
    Likes Received:
    0
    Trophy Points:
    0
    Re: CSV question

    Ooo, that's tough...how do you get anything done with these limitations?
    I use SAS as well, through either the Linux server (PuTTY as well), or my own machine. Although, there's no access or size restrictions.
    We also have this thing called WebDrive. We use it to make a virtual drive of the server in Windows. Considering that I'm not nuts about Linux, it's pretty useful for me.



     
  15. Dondrei

    Dondrei IncGamers Member

    Joined:
    Aug 1, 2005
    Messages:
    36,855
    Likes Received:
    30
    Trophy Points:
    467
    Re: CSV question

    I can do most things I need to just through Windows' interface with the drive, I mean all I really need most of the time is standard stuff like copying and pasting files, it's just those one or two really irritating little quirks that arise because Windows says tomahto and Unix says tomayto that cause really amazingly big problems.

    The guy who used to be my supervisor when I first joined (we've both since moved teams) is running a centralised automated reporting system built on SAS, all of which works amazingly well except for the export stage... because our business is a million years behind the times and still using Excel for its reports (don't get me started) and SAS's ability to export to Excel is extremely hampered because [long boring explanation of office politics goes here], anyway the upshot of it all is that he's using a Base SAS licence on his machine (even though everyone has shifted to EG now, God knows how much it costs the business to have just one Base licence for just one PC, Jesus) for no other purpose than to run exports, because Base can get around the barriers in place for EG (thereby making the whole logic of isolating EG invalid, but what the hey).

    Anyway, I could get into war stories but I've probably bored everyone enough for tonight.
     

Share This Page