Author Topic: 10@10 - Discussion of KFOG's show in general  (Read 68836 times)

ggould

  • Administrator
  • Master Poster
  • *****
  • Posts: 9159
    • View Profile
    • http://www.ggould.com
Re: 10@10 - Discussion of KFOG's show in general
« Reply #60 on: September 14, 2013, 10:12:37 PM »
I used to do simple database programming in dbase and 4D.  I'm probably too old to learn SQL.  It would be nice.
Don't stand in the way of LOVE!

Tinka Cat

  • The Core
  • Master Poster
  • *****
  • Posts: 8080
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #61 on: September 15, 2013, 12:09:20 PM »


Top artists from 2013:

artist   Count
Rolling Stones   23
Bruce Springsteen   19
U2   19
Beatles    13
Led Zeppelin   13
R.E.M.   12
David Bowie    12
Eric Clapton   12
The Who   12
The Beatles   10



tied for first!
~CPL593H~

Big Fingers McGee

  • The Core
  • Heavy Duty
  • *****
  • Posts: 3376
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #62 on: September 15, 2013, 02:00:05 PM »


Top artists from 2013:

artist   Count
Rolling Stones   23
Bruce Springsteen   19
U2   19
Beatles    13
Led Zeppelin   13
R.E.M.   12
David Bowie    12
Eric Clapton   12
The Who   12
The Beatles   10



tied for first!

Heh. I'm sure this is a common issue. For the record, I'd consider Bruce Springsteen/Bruce Springsteen & The E-Street Band to be one entity, as I would Tom Petty with and without the Heartbreakers, and so forth.

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #63 on: September 15, 2013, 09:05:10 PM »
love the idea of getting database together.  This is why I've always had a problem with misspellings in the post that contains the setlist.  Also why I don't like seeing "Crapton" in a list!
How about Sheryl 'Ho?
iirc, the Shraytabase was pulling from the KFOG setlist page, and not from here.

I have also been compiling my files from the set lists posted on the KFOG page.  I correct
spelling errors when I see them, but I have not been diligent about checking for them.  I also
have made no effort to use consistent names for the bands.

I can see using some kind of a spell check algorithm to locate mispelled and alternate group
names.
"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #64 on: September 15, 2013, 09:18:14 PM »
It's all a work in progress.  We don't necessarily find the optimum process the first time through, especially when I'm doing this after a round of golf and a few after golf adult beverages.  I'm basically hacking my way through the first time around.  It will get better, easier. 

I don't "know" SQL, but reading what you posted downthread makes it clear what you're
doing.  More comments on that in a reply to another post.

But, for now, a couple of results for 2013 (not QA'd very much_ --

Top artists from 2013:

artist   Count
Rolling Stones   23
Bruce Springsteen   19
U2   19
Beatles    13
Led Zeppelin   13

YES!!  Great stuff!  This is exactly the kind of thing I'm looking for.  All kinds of pointless trivia
can be generated.  (grin)

The first thought I had was to look for the Foo Fighters.  And they aren't on the list.  But they
probably would be for a list generated from AL's shows.  Which leads me to suggest that the
database should have a field for "host," so this kind of comparison can be made between DM,
AL, and RR etc.

Perhaps I should also send you the data for October through December 2012, so you have the
complete RR era through August.
"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #65 on: September 15, 2013, 09:21:33 PM »
Top artists by year:

I sorted and rearranged this a little...

year/count/artist

1966 3 Beach Boys
1966 3 Bob Dylan
1966 3 Donovan
1966 3 Rolling Stones

1967 4 Jimi Hendrix

1968 3 Moody Blues

1969 3 Led Zeppelin
1969 3 Rolling Stones

1971 3 The Who

1973 4 Led Zeppelin
1973 3 Dr. John
1973 3 Elton John
1973 3 Jimmy Cliff

1974 2 David Bowie

1975 3 Bruce Springsteen
1975 3 Led Zeppelin
1975 3 Steely Dan

1977 3 Eric Clapton

1978 3 George Thorogood

1980 2 Dire Straits

1983 4 The Police
1983 3 Eurythmics

1984 3 Bruce Springsteen
1984 3 Joe Jackson
1984 3 Prince
1984 3 Thompson Twins
1984 2 Don Henley

1985 3 John Fogerty

1986 3 Fabulous Thunderbirds
1986 3 The Smiths

1987 4 U2
1987 3 INXS

1988 3 Escape Club
1988 3 Fine Young Cannibals
1988 3 Steve Winwood
1988 3 U2

1991 3 Dire Straits
1991 3 Stevie Ray Vaughan
1991 2 Divinyls

1993 3 Chris Isaak
1993 3 Cracker
1993 3 John Hiatt
1993 3 Sheryl Crow

1994 3 Blues Traveler

1995 3 Garbage
1995 3 Radiohead

1997 3 Chumbawamba
1997 3 Third Eye Blind

NULL 3 Beatles
NULL 3 Bruce Springsteen
"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #66 on: September 15, 2013, 09:26:43 PM »
Of course, if dischead and others want to get together to throw a Frisbee around, that would be fun too.  I miss Frisbee.

And Frisbee misses you!  Come back... come back to Frisbee!  All will be forgiven!


Did you know there's a permanent, 18-hole disc golf course in Golden Gate Park?  You can have
Frisbee and golf too!

"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #67 on: September 15, 2013, 09:41:21 PM »
I've started on the 2002 list dischead provided.  Ran into some issues, and I've stopped for the day. I think tomorrow I might try from a different angle.  I can do everything with sql, but perhaps reading files sequentially with a program would be easier.  I need to download some software.

I can see what you're doing with the SQL code, and while you probably could do everything
that way, I'm guessing SQL isn't very good at string manipulation.  So writing a program to
pre-process my text files and massage them into something that can more easy be slurped
directly into a database is probably the way to go, and is what I had envisioned.  I tried to
edit my text files into a consistent format to ease that process.

I'm interested in this project whether anyone ends up benefiting or not.  I think I'll learn a new language to get it to work.  Just for fun.

Well, I use BASIC because it has fairly good string handling, or C if I need to get more into
character nitty-gritty.  Or even TECO -- that's how old-school I am.  But those choices are
largely because those are the tools I have and I'm very familiar with them.  From what I've
heard, Perl might be a good choice, and Python is the current darling of the computing in-crowd.
"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #68 on: September 15, 2013, 09:53:14 PM »
Top artists from 2013:

artist   Count
Rolling Stones   23
Bruce Springsteen   19
U2   19
Beatles    13
Led Zeppelin   13
R.E.M.   12
David Bowie    12
Eric Clapton   12
The Who   12
The Beatles   10
tied for first!

Nice catch... but what you don't know -- because the list is truncated -- is that "The Rolling Stones"
have four songs.  So they're still number one.

This nicely illustrates that any leading "the" in artist names will have to be removed when loaded
into the database.  Case should be ignored as well.  Punctuation is trickier; the periods in R.E.M.
should probably be removed, but not hypens or apostrophes.
"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #69 on: September 15, 2013, 10:02:50 PM »
Top artists from 2013:

artist   Count
Rolling Stones   23
Bruce Springsteen   19
U2   19
Beatles    13
Led Zeppelin   13
R.E.M.   12
David Bowie    12
Eric Clapton   12
The Who   12
The Beatles   10
tied for first!

Nice catch... but what you don't know -- because the list is truncated -- is that "The Rolling Stones"
have four songs.  So they're still number one.

This nicely illustrates that any leading "the" in artist names will have to be removed when loaded
into the database.  Case should be ignored as well.  Punctuation is trickier; the periods in R.E.M.
should probably be removed, but not hypens or apostrophes.

And I just looked... "Who" has one song, so they jump ahead to tie with Led Zeppelin.
"Your favorite songs, played beautifully"

Lightnin' Rod

  • Administrator
  • Heavy Duty
  • *****
  • Posts: 4504
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #70 on: September 16, 2013, 09:08:27 AM »

Well, I use BASIC because it has fairly good string handling, or C if I need to get more into
character nitty-gritty.  Or even TECO -- that's how old-school I am.  But those choices are
largely because those are the tools I have and I'm very familiar with them.  From what I've
heard, Perl might be a good choice, and Python is the current darling of the computing in-crowd.

I was thinking Python.  Perl has excellent string manipulation capabilities, but it's been a number of years since I've used it.
and any fool knows
a dog needs a home
a shelter
from pigs on the wing

Lightnin' Rod

  • Administrator
  • Heavy Duty
  • *****
  • Posts: 4504
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #71 on: September 17, 2013, 09:07:23 PM »

And I just looked... "Who" has one song, so they jump ahead to tie with Led Zeppelin.

Where do people stand on ampersands in band names? 
I believe Big Head Todd and the Monsters is correct, not Big Head Todd & the Monsters.
However, I do believe Crosby, Stills & Nash is correct -- not "and".

My desire would be to replace "&" with "and" every time, but I guess we can not.   :-\


ETA:
Crosby, Stills & Nash
Crosby, Stills, & Nash
Crosby Stills & Nash
CSN
?
maybe we should dump commas too
« Last Edit: September 17, 2013, 09:26:25 PM by Lightnin' Rod »
and any fool knows
a dog needs a home
a shelter
from pigs on the wing

Lightnin' Rod

  • Administrator
  • Heavy Duty
  • *****
  • Posts: 4504
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #72 on: September 17, 2013, 09:55:35 PM »
After some cleanup (see the end) a better, but not perfect yet, list for 2013.

Rolling Stones   27
Beatles   23
Bruce Springsteen   20
U2   19
Who   15
Kinks   14
Led Zeppelin   13
R.E.M.   12
David Bowie   12
Eric Clapton   12
Pretenders   11
Grateful Dead   11
Fleetwood Mac   10
Doobie Brothers   10
Elton John   10
Bonnie Raitt   10
Dire Straits   9
Police   9
Neil Young   9
Stevie Ray Vaughan   9
Steve Winwood   8
Santana   8
Steely Dan   8
Paul Simon   8
Red Hot Chili Peppers   8
Joe Jackson   8
Little Feat   8
Beach Boys   8

-- cleanup
-- get rid of leading 'The ' (but not for Them, or from 'The The' who were not present anyway.
update setEntries set artist = right(artist, LEN(rtrim(artist)) - 4) from setEntries where left(artist, 4) = 'The '

-- bad parsing because of dashes in artist name (I can fix earlier) though there should not be a dash in B52's
(should there be an apostrophe?)
update setEntries set song = 'Loveshack', artist = 'B52''s' where setEntriesId = 309
update setEntries set song = 'Take on Me', artist = 'A-Ha' where setEntriesId = 1696
update setEntries set song = 'You Ain''t Seen Nothin'' Yet', artist = 'Bachman-Turner Overdrive' where setEntriesId = 799
update setEntries set song = 'Let It Ride', artist = 'Bachman-Turner Overdrive' where setEntriesId = 799

-- various issues, should be obvious (BTO, STP, CCR and CSN not spelled out, general mis-spellings, other shortcuts)
update setEntries set artist = 'Bachman-Turner Overdrive' where artist = 'BTO'
update setEntries set artist = 'Stone Temple Pilots' where artist = 'STP'
update setEntries set artist = 'Alanis Morissette' where artist = 'Alanis Morisette'
update setEntries set artist = 'Three Dog Night' where artist = '3 Dog Night'
update setEntries set artist = 'Allman Brothers Band' where artist = 'Allman Brothers'
update setEntries set artist = 'Big Head Todd and the Monsters' where artist = 'Big Head Todd & the Monsters'
update setEntries set artist = 'Bob Marley & the Wailers' where artist = 'Bob Marley & Wailers'
update setEntries set artist = 'Bruce Springsteen' where artist = 'Bruce Springteen'
update setEntries set artist = 'Simon & Garfunkel' where artist = 'Simon & Garfunkle'
update setEntries set artist = 'Commander Cody & His Lost Planet Airmen' where artist = 'Cody & His Lost Planet Airmen'
update setEntries set artist = 'Dave Clark Five' where artist = 'Dave Clark 5'
update setEntries set artist = 'Dire Straits' where artist = 'Dire Strats'
update setEntries set artist = 'Crosby, Stills, & Nash' where artist = 'Crosby Stills & Nash' or artist = 'CSN'
update setEntries set artist = 'Creedence Clearwater Revival' where artist = 'CCR'
-- one backward entry in rawdata
update setEntries set artist = 'Prince', song = 'Little Red Corvette' where artist = 'Little Red Corvette'
and any fool knows
a dog needs a home
a shelter
from pigs on the wing

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #73 on: September 17, 2013, 11:55:22 PM »
And I just looked... "Who" has one song, so they jump ahead to tie with Led Zeppelin.
Where do people stand on ampersands in band names? 
I believe Big Head Todd and the Monsters is correct, not Big Head Todd & the Monsters.
However, I do believe Crosby, Stills & Nash is correct -- not "and".

My desire would be to replace "&" with "and" every time, but I guess we can not.   :-\

I would guess we can... it's "our" db, after all.  My argument is that "and" and ampersand are
equivalent, and while some artists may choose to use one or the other consistently for their
own stylistic reasons, we don't necessarily have to follow their lead, particularly if it results in
much wailing and gnashing of teeth while attempting to construct a useful database.  And the
parade of people posting the set lists on the KFOG site aren't consistent either, nor do they
always conform to what the artist does.

ETA:
Crosby, Stills & Nash
Crosby, Stills, & Nash
Crosby Stills & Nash
CSN
?
maybe we should dump commas too

Well, CSN (and also CSNY, DMB, SRV, etc.) is an abbreviation and should be expanded.  I regard
them as spelling errors, and change them when I notice them.  (See my comments in the next
post.)  As for commas, the easiest solution would be to strip them out -- again, the argument
being that we wish to minimize the rending of garments in what is supposed to be a enjoyable
enterprise.  More complicated possibilities include using a special string comparison routine that
ignores punctuation, or storing both the original string for display purposes and a processed
string (i.e., with punctuation removed) for comparisons.

And then there's "Iggy Pop & Kate Pierson" vs. "Iggy Pop with Kate Pierson," and both "REM"
and "REM with Kate Pierson" are listed as performing "Shiny Happy People."  (REM also does
"Shiny, Happy People" -- another reason to dump commas.)

BTW, for the non-computer people who are reading, this is the kind of niggling detail that
computer people often find themselves caught up in when attempting to write a program that
does even a relatively simple task that is easy and obvious for a human.  I have read long and
involved monographs on how to account for all the possible variations in people's names, to
say nothing of street addresses.  And that's just in the United States.  When writing a program
for an international audience, there is a mind-boggling number of different address conventions,
telephone number formats, date and time formats, currency designations, etc.  which have to
be accommodated.
"Your favorite songs, played beautifully"

dischead

  • The Core
  • kiloposter
  • *****
  • Posts: 1373
  • Honorary Knight of the Command Line
    • View Profile
Re: 10@10 - Discussion of KFOG's show in general
« Reply #74 on: September 18, 2013, 12:41:50 AM »
-- cleanup
-- get rid of leading 'The ' (but not for Them, or from 'The The' who were not present anyway.
update setEntries set artist = right(artist, LEN(rtrim(artist)) - 4) from setEntries where left(artist, 4) = 'The '

Yes, although technically "The The" should be "The" if we're going to be consistent about dropping
leading the's.  But I'll accept an exception in this case.  They do appear in several years.


-- bad parsing because of dashes in artist name (I can fix earlier) though there should not be a dash in B52's
(should there be an apostrophe?)
update setEntries set song = 'Loveshack', artist = 'B52''s' where setEntriesId = 309
update setEntries set song = 'Take on Me', artist = 'A-Ha' where setEntriesId = 1696
update setEntries set song = 'You Ain''t Seen Nothin'' Yet', artist = 'Bachman-Turner Overdrive' where setEntriesId = 799
update setEntries set song = 'Let It Ride', artist = 'Bachman-Turner Overdrive' where setEntriesId = 799

A hyphen in B52's is a spelling error.  (See below.)  According to Wikipedia, the apostrophe should
be there until 2008, when the band dropped it from their name.  As for other hyphens, one of
the things I have been careful about in my files is making sure that the song title is separated
from the artist by space hyphen space (" - ").  This convention isn't always observed by the
people posting the set lists on the KFOG site, but I try to correct it immediately.  If I've missed
any, it's my fault, and it should be corrected as with spelling errors.  This means that "legitimate"
hyphens in artist names and song titles should always be between two alphanumeric characters.


-- various issues, should be obvious (BTO, STP, CCR and CSN not spelled out, general mis-spellings, other shortcuts)

It's great that you're catching these and can easily write an update that fixes them, but in my
view these are errors that I should (and will) fix in my files.  As you refine your process and
I correct my data, eventually they will be eliminated and this kind of ad hoc code won't be
necessary.  I would suggest that every so often when you've collected a bunch of new errors,
send them to me in a PM so as not to bore everyone else, and I'll update my files.  Periodically
I'll send you the updated files.
"Your favorite songs, played beautifully"