>> GOOD AFTERNOON EVERYONE AND
WELCOME TO E-UTILITIES FOR
PUBMED.
MY NAME IS MIKE DAVIDSON.
I'M A LIBRARIAN IN BETHESDA,
MARYLAND HERE WITH REX ROBISON.
EACH OF US HAVE DIFFERENT JOBS
AT THE LIBRARY OF MEDICINE.
WE'RE ALL MEMBERS OF THE NLM
TRAINING TEAM AND ANOTHER THING
WE ALL DO IS WE ARE ALL ON THE
NLM PUBMED CUSTOMER SERVICE
TEAMS SO WE MAY BE THE ONES
WRITING BACK TO YOU IF YOU'VE
WRITTEN US.
BEFORE WE GET INTO TODAY'S
AGENDA LET ME START OFF BY
SAYING WE KNOW OUR AUDIENCE HAS
LOTS OF DIFFERENT LEVELS OF
EXPERIENCE.
WE EXPECT YOU ALL HAVE SOME
BACKGROUND SEARCHING AND WORKING
IN PUBMED BUT KNOW SOME MAY HAVE
MORE TECHNICAL EXPERTISE THAN
OTHERS.
WE ARE TRYING TO OFFER SOMETHING
FOR EVERYONE.
IF YOU'RE MORE TECHNICALLY
INCLINED AND EXPERIENCE AND IT'S
FAMILIAR HANG ON THERE'LL BE
MORE ADVANCED STUFF TOWARDS THE
END AND IF YOU ARE LESS
EXPERIENCED STICK WITH US
BECAUSE WE HAVE MORE EXPLANATORY
STUFF AS WELL.
WE'LL START OFF TODAY'S CLASS BY
TAKING A LOOK AT PARTICULARLY
CHALLENGES PUBMED QUESTIONS AND
INTRODUCE YOU TO A SOLUTION AND
PROVIDE A BACKGROUND ON HOW THE
SOLUTION WORKS TO GIVE A SOLID
FOUNDATION AND OFFER PRACTICAL
TOOLS FOR THE REAL WORLD WHICH
WILL HOPEFULLY ANSWER YOUR
QUESTIONS.
NOW, AS I SAID AT THE BEGINNING
ONE OF MY JOBS HERE AT THE
NATIONAL LIBRARY OF MEDICINE IS
ON THE PUBMED TEAM AND WE'VE
SEEN AN INCREASE IN A CERTAIN
TYPE OF QUESTIONS THAT ALL TAKE
THE SAME GENERAL STRUCTURE.
THAT STRUCTURE IS HOW CAN I GET
A PARTICULAR SET OF DATA FROM
PUBMED IN A CERTAIN FORMAT.
HERE'S AN EXAMPLE OF A QUESTION.
WHICH AUTHORS HAVE A LOT OF
PUBLICATIONS ON A SPECIFIC
SUBJECT.
WHO'S HOT IN A PARTICULAR FIELD.
WE DON'T JUST WANT AUTHORS WITH
A LOT OF PUBLICATION OVERALL BUT
A LOT OF RECENT PUBLICATIONS.
WHICH ORGANIZATIONS ARE FUNDING
RESEARCH ON THE TOPIC.
I WANT THE DATA IN A FORMAT OR
ANALYZE OR VISUALIZE LIKE
MICROSOFT XCEL.
THIS SAY CHALLENGE BECAUSE WHILE
YOU CAN DOWNLOAD THE FORMAT YOU
HAVE TO SEPARATE THEM TO DO ANY
ANALYSIS.
IT DOESN'T INCLUDE FUNDING
INFORMATION SO YOU NEED THAT
DATA IN A DIFFERENT WAY.
SAY YOU'RE TRYING TO VISUALIZE
TREND IN RESEARCH ON SWINE FLU
H1N1 AND IF YOU LOOK AND THE
RIGHT SIDE YOU'LL SEE WHAT WE
CALL THE DISCOVERY BAR WHICH
CONTAINS USEFUL INFORMATION TO
EXPAND AND PROVIDE CONTEXT FOR
SEARCH RESULTS.
YOU'LL SEE A RESULTS BY YEAR
HISTOGRAM.
THIS SHOWS HOW MANY WERE
PUBLISHED PER YEAR AND YOU SEE
SOME IN 2009 AND 2010 AND TRAILS
OFF GIVEN THE PUBLIC.
IF IT'S LIVE DEMO INSTEAD OF A
SCREEN SHOT WE CAN PULL THE DATA
BEHIND THE GRAPHIC TO USE
HOWEVER WE WANT.
SAY I WANT TO DO A MORE SPECIFIC
SEARCH FOR H1N1 ASTHMA.
THE RESULT SET IS TOO SMALL.
IF AWAY WANT TO GET AT THAT DATA
WE HAVE TO DO THAT IN ANOTHER
WAY.
NOW, YOU HAVE KIND OF ADVANCED
QUESTIONS.
THE MAJORITY CAN GO TO
WWW.PUBMED.com BUT AS WE'VE SEEN
FOR CERTAIN JOBS ADVANCE USERS
LIKE YOU SOMETIMES NEED
DIFFERENT TOOLS.
IT'S ABOUT HAVING THE RIGHT TOOL
FOR THE JOB.
SO AS ADVANCED USERS WHAT DO YOU
NEED YOUR TOOLS TO DO?
YOU NEED THEM TO GET EXACTLY THE
DATA YOU NEED AND ONLY THE DATA
YOU NEED IN THE SPECIFIC OUTPUT
FORMAT YOU NEED IT IN.
AND SO TO THAT END WE WANT TO
INTRODUCE YOU TO E-UTILITIES.
IT'S AN API FOR ACCESSING PUBMED
AND OTHER NCBI DATABASES.
THAT'S GREAT BUT WHAT'S IT MEAN?
IF YOU NEVER USE AN API BEFORE
OR MAYBE YOU HAVE USED ONE BUT
STILL NOT 100% WHAT AN API IS
LET'S START WITH A DEFINITION.
IT STANDS FOR APPLICATION
PROGRAMMING INTERFACE.
IT'S A SET OF TOOLS, ROUTINES
AND PROTOCOLS FOR BUILDING
SOFTWARE APPLICATIONS.
THAT'S A GOOD START BUT AS A
DEFINITION IT'S STILL PRETTY
GENERIC.
LET'S GET MORE SPECIFIC ABOUT
THE E-UTILITIES API WHICH SETS
UP ROUTINES AND PROTOCOLS THAT
ALLOWS YOU TO INTERACT WITH THE
DATA IN OVER TWO DOZEN
DATABASES.
OK, BUT WHAT DOES THAT MEAN?
IT'S SIMPLER THAN IT SOUNDS.
THE UTILITIES API IS A SERIES OF
RULES FOR QUERYING A DATABASE.
FOR TODAY'S CLASS THE DATABASE
WE'LL BE TALKING ABOUT IS
PUBMED.
AS I MENTIONED THE RULES WORK
FOR QUERYING A BUNCH OF NCBI
DATABASE.
THAT'S ONE OF THE DIVISION THE
NATIONAL LIBRARY OF MEDICINE.
THEY DO A LOT OF DIFFERENT
THINGS.
ONE JOB IS DEVELOPING AND
MAINTAINING A VARIETY OF
DATABASES OF MEDICAL INFORMATION
INCLUDING PUBMED AND DEVELOPED
THE E-UTILITIES API AS A WAY OF
QUERYING THE DATABASES.
YOU CONSTRUCT THEM IN THE FORM
OF STRUCTURED URLs.
EACH ASKS THE NCBI FOR
INFORMATION AND RETURNS IT IN A
FORMAT.
WHAT INFORMATION YOU GET AND
WHAT FORMAT YOU GET IT DEPENDS
ON HOW YOU CONSTRUCT THE URL.
YOU YOURSELF PROBABLY WILL NOT
BE CONSTRUCTING THEM ON A
REGULAR BASES.
IN A MINUTE I'LL TELL YOU HOW TO
FORM THE TOOLS TO CREATE THEM
FOR YOU BUT IN ORDER TO
UNDERSTAND HOW IT WORKS AND WHAT
THEY'RE DOING WE'LL GO OVER THE
BASICS OF CREATING AN
E-UTILITIES URL.
I MAY SEEM DAUNTING AT FIRST
BECAUSE THEY'RE DESIGNED TO ASK
A QUESTION TO THE SERVER AND
THERE'S AN INFINITE NUMBER OF
QUESTIONS YOU CAN ASK AND URLs
YOU CAN CREATE.
HOWEVER, ALL THE URLs FOLLOW THE
SAME SET OF RULES AND SAME BASIC
PATTERN.
LET'S A TAKE A LOOK.
WE'RE GOING INTO MORE DETAIL
SOON BUT IN SHORT HAVE YOU THE
BASE URL WHICH IS THE ADDRESS OF
THE E-UTILITIES SERVER AND THE
DETAILS OF YOUR QUERY.
IN PRACTICE THEY LOOK SOMETHING
LIKE THIS.
THE BASE URL IS THIS CHUNK HERE.
THAT'S ALWAYS THE SAME.
THE UTILITY NAME WILL BE ONE OF
NINE OPTION LOOKS SOMETHING LIKE
THIS AND THEY'LL HAVE A DOT-FCGI
EXTENSION AND WE'LL TALK ABOUT
THAT AND THE PARAMETERS THAT
DEFINE THE DETAIL OF YOUR QUERY.
THEY MAY LOOK SCARY AT THE
BOTTOM BUT ONCE YOU LEARN THE
SYNTAX YOU'LL SEE FAMILIAR
SEARCH TERMS AND TAGS AND IT
WILL START GETTING EASY FOR
SOMEONE WHO SPENT ANY TIME USING
PUBMED.
WE HAVE A JOURNAL SEARCH HERE TO
SEARCH THE JOURNAL FIELD AND
DON'T GET TURNED OFF BY THE
PLUSES.
THEY'RE IN SPACE OF SPACES SINCE
YOU CAN'T HAVE A SPACE IN THE
URL.
WE'LL GET INTO MORE DETAIL THE
MOMENT.
FIRST IS THE BASE URL.
THAT'S ALWAYS THE SAME.
THIS IS THE BEGINNING OF EVERY
E-UTILITIES URL AND SERVER.
THIS IS IT.
NOT MUCH MORE TO SAY ABOUT THE
BASE URL.
IT'S AN ALWAYS THERE AND THE
SAME.
WE HAVE ONE PART OUT OF THE WAY.
THE SECOND PART IS CHOOSING
WHICH ONE OF THE NINE SPECIFIC
UTILITIES YOU'LL BE USING.
EACH OF THE UTILITIES ASKS A
DIFFERENT KIND OF QUESTION.
WHICH YOU CHOOSE WILL DEPEND ON
WHICH QUESTION YOU WANT TO ASK.
FOR EXAMPLE, E-SEARCH SEARCHING
THE DATABASE AND LOOKS FOR
UNIQUE IDENTIFIERS.
IF YOU'RE SEARCHING PUBMED IT
RETURNS A LIST OF PMIDs.
IT'S IMPORTANT TO NOTE E SEARCH
DOESN'T RETRIEVE FULL REPORTS.
TO DO THAT YOU USE A DIFFERENT
UTILITY.
YOU USE E-FETCH.
THAT TAKES UNIQUE IDENTIFIERS
LIKE FROM AN E-SEARCH AND
RETRIEVES THE FULL RECORD
INCLUDING ALL THE DATA IN ALL
THE FIELDS FOR EACH OF THOSE
UIDs.
NOTE THE RECORD YOU'RE
RETRIEVING IS THE SAME AT PUBMED
SO THERE'S NO DIFFERENCES IN THE
CONTENT BETWEEN THE TWO
CHANNELS.
>> THERE'S ALSO SEVEN OTHER
UTILITIES THAT DO DIFFERENT
THINGS BUT WE WON'T GET INTO
THEM NOW.
IF YOU WANT TO SEE THE FULL LIST
CHECK OUT THE WEBSITE.
ALL RIGHT.
SO WE'VE DONE TWO PARTS OF THE
E-UTILITIES URL NOW THE THIRD
PART.
AS I MENTIONED BEFORE THE LAST
PART OF AN E-UTILITIES URL IS
THE PARAMETERS WHICH DEFINE THE
DETAILS OF YOUR QUERY.
WHICH PARAMETERS YOU CAN USE
VARY BY UTILITY.
NOT ALL PARAMETERS APPLY TO ALL
UTILITY.
SO ARE REQUIRED AND SOME AREN'T.
ONE PARAMETER YOU'LL SEE IN OF
THE IS THE DB PARAMETER.
IT SPECIFIES WHICH DATABASE
YOU'LL BE QUERYING.
REMEMBER, YOU CAN QUERY A NUMBER
NOT JUST PUBMED.
IF YOU USE E-SEARCH YOU PROBABLY
WANT A SEARCH PARAMETER WHICH
DEFINES YOUR SEARCH STRENGTH.
OTHERS INCLUDE THE I.D.
PARAMETER WHICH LETS YOU LIST A
SPECIFIC IDENTIFIERS LIKE PMIDs.
THE PARAMETERS WHICH DEFINES
WHICH FORMAT YOU WANT YOUR
RESULTS IN I'LL SHOW AN EXAMPLE
OF LATER ON AND TONS MORE I HAVE
TIME TO SHOW YOU NOW AND THERE'S
MORE AT OUR WEBSITE.
SO WE TALKED ABOUT THE THREE
PARTS.
LET'S PUT THEM TOGETHER.
WITH START WITH THE BASE URL AND
ADD THE UTILITY, E SEARCH.
WE HAVE THE DOT-FCGI EX TENSE
AND WE HAVE THE DETAILS OF OUR
QUERY.
OUR DB AND THE PLUSES ARE IN
SPACE OF SPACES SO DON'T GET
THROWN OFF BY THOSE.
ONCE WE PUT THE THREE PARTS
TOGETHER WE HAVE A UTILITIES
URL.
NOW THAT WE'VE CREATED THE URL
WE CAN PUT IT IN ANY WEB BROWSER
AND SEE WHAT HAPPENS.
I'LL HOP OUT TO WEB BROWSER IN A
MOMENT AND KATIE WILL PUT THE
URL IN THE CHAT BOX AND FEEL
FREE TO PUT IT IN YOUR OWN
BROWSER BUT COPY AND PASTE THEM.
WEBEX DOESN'T LIKE THEM IN
BRACKETS FOR SOME REASON.
I'LL GET THE URL COPIED AND
PASTED HERE.
YOU MAY HAVE DIFFICULTY SEEING
THAT BECAUSE IT'S PRETTY SMALL.
THEN I JUST HIT ENTER AND GET
THE RESULTS OF MY E-SEARCH,
E-UTILITIES URL.
A COUPLE THINGS TO NOTE.
WE START WITH A COUNT, HOW MANY
RESULTS MATCHED OUR SEARCH TERMS
AND A LIST OF OUR PMIDs.
REMEMBER IT RETRIEVES THEM ONLY
NOT FULL RECORDS.
THESE ARE QUERY TRANSLATION LIKE
YOU FIND IN THE WEB VERSION OF
PUBMED.
ALL THE QUERY TRANSLATION AND
AUTOMATIC TERM MAPPING ALSO
HAPPENS WITH E SEARCH AND THAT
TRANSLATION IS EVIDENT HERE AND
THE QUERY IS DETERMINED BY OUR
PARAMETERS.
IN THIS CASE WE'RE SEARCHING FOR
ARTICLES IN BREAST CANCER
PUBLISHED IN 2008.
TRY GOING TO THE URL BAR AND
CHANGE SCIENCE TO NATURE.
WE'RE KEEPING THE REST OF THE
SEARCH STREAM AND THEN I GET
DIFFERENT RESULTS BECAUSE THESE
ARE THE JOURNAL OF NATURE RATHER
THAN THE JOURNAL OF SCIENCE.
SO THAT THE TWO E-SEARCH
EXAMPLE.
NOW AN E-FETCH EXAMPLE.
WE'LL TAKE A LOOK.
WE HAVE OUR BASE URL THE SAME AS
ALWAYS, WE HAVE CHANGED OUR
UTILITY FROM E-SEARCH TO
E-FETCH.
WE'LL BE RETRIEVING RECORDS AND
OUR PARAMETERS IS TO PROVIDE
DETAILS.
THE UIDs FOR THE RECORDS WE WANT
TO RETRIEVE AND THE FORMAT WE
WANT TO RETRIEVE THEM IN.
WHEN I GO BACK OUT TO THE
BROWSER WHICH KATE WILL PUT IN
THE CHAT BOX -- WHEN I PUT THAT
IN AND HIT ENTER I RETRIEVE THE
TWO RECORDS I SPECIFIED IN MY
PARAMETER AND RETRIEVED IT IN
THE TEXT FORMAT IN THE ABSTRACT
VIEW OF THE TEXT.
IT'S THE SAME ABSTRACT VIEW YOU
CAN GET IN THE WEB VERSION OF
PUBMED.
THE DATA IS ALSO THE SAME.
WE'RE RETRIEVING THE TWO RECORDS
AND WE CAN CHANGE THE PARAMETERS
TO CHANGE THE FORMAT AND VIEW.
IF I CHANGE THEM I'M GETTING THE
SAME TWO RECORDS BUT IN A
DIFFERENT FORMAT.
NOW I'M SEEING THEM IN FULL X
ACT.
IF ROOK AT THE ABSTRACT WE SEE
THEY'RE NOT THE SAME TWO
RECORDS.
THEY'RE JUST BEING DISPLAYED IN
A DIFFERENT FORMAT.
NOW WE SPENT TEN, 15 MINUTES TO
ACCOMPLISH WHAT WE CAN DO IN
P.M. -- PUBMED FOR SOME REASON
GOING TO WWW.PUBMED.com IS THE
RIGHT CHOICE.
WHERE IT'S STRONGEST IS MULTIPLE
URLs TO CREATE THE NEXT.
I'LL PUT ANOTHER E-SEARCH URL IN
MY BROWSER.
THIS IS THE ASTHMA SEARCH FROM
BEFORE.
NOW, IDEAL WILLY IF WE'RE GOING
BACK TO THE HISTOGRAM WE WANT TO
RETRIEVE THE FULL RECORD TO LOOK
AT THE PUBLICATION DATA AND
CREATE THE HISTOGRAM.
IN ORDER TO GET THE FULL RECORDS
WE NEED TO E-FETCH.
I'M GOING TO CREATE THE E-UNITED
STATES
E-UTILITIES AND WE NEED TO
CHOOSE OUR UTILITY.
WE'LL CHOOSE E-FETCH AND I HAVE
TO USE DOT-FCGI WITH THE
EXTENSION ON THE END AND ADD OUR
PARAMETERS, PB EQUALS PUBMED AND
OUR AMPERSAND AND SPECIFY WHAT
WE WANT TO FETCH.
I'LL COPY THE PMID AND PASTE IT
THERE AND PUT A COMMA AND SLIDE
THIS IN THIS AND PUT A COMMA AND
THIS IS CLEARLY RIDICULOUS.
THERE'S NO WAY THIS IS A
PRACTICAL SOLUTION.
BECAUSE OF THAT IS WHY YOU WANT
TO USE E-UTILITIES IN A
PROGRAMMING ENVIRONMENT.
BY USING IT WE START TO UNLOCK
THE REAL POWER OF THE API.
WE DON'T HAVE TO CREATE THE URLs
BY MAND.
WE CAN WRITE THE RULES RIGHT IN
THE PROGRAM AND TAKE THE RESULTS
OF ONE QUERY TO GENERATE THE
NEXT ONE.
CREATING AN E-FETCH URL FROM OUR
E-SEARCH RESULTS.
WE CAN MANIPULATE THE OUTPUT
PULLING OUT JUST THE PUBLICATION
RATE FROM THE RECORD AND
COLLATING THE DATA TO CREATE THE
HISTOGRAM.
ORDINARILY TO USE E-UTILITIES
THE PROGRAMMING ENVIRONMENT
YOU'D NEED TO COMPLETE A SERIES
OF COMPLICATED TIME-CONSUMING
STEPS BEFORE YOU GOT CLOSE TO
ANSWERING YOUR QUESTION.
WAIT, WE'VE DONE A LOT OF THAT
FOR YOU.
A TOOL KIT EXISTS IN THE FORM OF
E DIRECT.
THAT'S SOFTWARE DEVELOPED BY
NCBI.
THEY SET UP TOOLS WITH THE
CREATION RULES BUILT IN.
AND YOU CAN QUERY THE SERVER AND
GET THE RESULTS AND YOU CAN
EXTRACT THE ELEMENTS YOU NEED
FROM THE PUBMED AND IT DOESN'T
TAKE MUCH TO GET STARTED BUT
WORKS WITH OTHER TOOLS AND
PROGRAMMING LANGUAGES SO
ADVANCED USERS CAN BUILD MORE
ADVANCED AND POWERFUL RESULTS.
THE FIRST YOU'LL NEED IS ACCESS
TO A UNITS COMMAND TERMINAL.
IF YOU'RE USE MAC YOU HAVE IT
READY TO GO.
IF YOU'RE USING WINDOWS LIKE I
AM YOU CAN INSTALL THE FREE
UNITS TERMINAL EMULATOR THAT
WORKS IN A WINDOWS ENVIRONMENT.
IN ADDITION YOU'LL NEED TO
INSTALL THE E-DIRECT SOFTWARE
AND THERE'S A PREWRITTEN SCRIPT
THAT WORKS IN YOUR TERMINAL
WHICH DOWNLOADS, INSTALLS AND
CONFIGURES E-DIRECT.
IT DOESN'T TAKE TOO MUCH PRIOR
KNOWLEDGE TO LEARN BUT HELPS TO
HAVE SOME PRIOR KNOWLEDGE TO
WORK YOUR WAY AROUND OR TAKE MY
ROUTES WHICH IS HAVING
KNOWLEDGEABLE TRENDS TO PESTER
WITH QUESTIONS.
EVEN IF YOU DON'T HAVE SOMEBODY
LIKE THIS THERE'S A LOT OF HELP
ON LINE AND GO TO YOUR SEARCH
ENGINE LOOK AT UNIX AND SOMEBODY
HAS PROBABLY ASKED THE SAME
QUESTION AND GOT AN ANSWER YOU
CAN USE.
A BIG ADVANTAGE OF E-DIRECT IS
EXTRACT DATA FROM THE PUBMED
FORMAT.
IN ORDER TO TAKE ADVANTAGE OF
THIS YOU CAN LOOK AT THE
STRUCTURE.
IF YOU LOOK AT IT YOU'RE GOOD GO
BUT YOU CAN ALWAYS GET TO THE
XML VIA THE FORMAT MENU IN THE
WEB VERSION OF PUB MED.
LOOKING AT THE DATA THERE AND
HOW IT'S ARRANGED HELPS YOU GET
A FEEL FOR THE STRUCTURE OF THE
DATA AND IT CAN BE HELPFUL TO
HAVE PROGRAMMING KNOWLEDGE PRO
OR PYTHON BUT IT'S NOT REQUIRED.
I WILL SAY THE MORE YOU KNOW
ABOUT UNIX OR XML OR PROGRAMMING
THE MORE YOU'LL DEAL WITH
E-DIRECT MORE QUICKLY.
WITH THOSE PRELIMINARIES OUT OF
THE WAY LET'S LOOK AT WHAT
E-DIRECT LOOKS IN ACTION.
IT WORKS IN AN UNIX ENVIRONMENT.
IF I WANTED TO SEARCH PUBMED I'D
TYPE AN E-SEARCH COMMAND FOR
H1N1 ASTHMA AND GET MY RESULT.
WE KNOW WE HAVE PEOPLE ATTEND A
WIDE RANGE OF EXPERIENCES AND
WANT TO GIVE THOSE WITH LESS
PROGRAMMING EXPERIENCE FAIR
WARNING.
AS YOU'VE SEEN A LITTLE BIT IN
THE NEXT COUPLE MINUTES WE'LL
START SHOWING YOU THINGS AND
THROWING AROUND TERMS THAT MAY
BE SCARY OR OUTSIDE YOUR COMFORT
ZONE.
PLEASE DON'T PANIC.
EVEN IF YOU'RE WILLING TO PUT IN
A LITTLE BIT EFFORT I PROMISE
YOU YOU'LL BE ABLE TO PICK THIS
STUFF UP AND PUT IT TO USE.
OVER THE NEXT COUPLE SLIDES
WE'LL SHOW YOU EXAMPLES OF AN E
UTILITIES AND E-DIRECT IN
ACTION.
IF YOU HAVE MORE OF A TECHNICAL
BACKGROUND SOME OF THIS STUFF
WILL BE OLD NEWS TO YOU AND
YOU'LL SEE YOU CAN GET STARTED.
IF IT'S NEWER TO YOU FOR NOW
PICK A BROADER VIEW OF THE STUFF
WE CAN DO WITH E-UTILITIES.
IF IT SEEMS MORE USEFUL YOU CAN
TAKE A HANDS-ON CLASS OR READ
THE DOCUMENTATION ON THE WEBSITE
OR FIND THE MORE TECHNICALLY
INCLINED PERSON AT YOUR
ORGANIZATION AND HAVE THEM HELP
YOU THROUGH THE EARLY STAGES.
LET'S LOOK AT HOW IT CAN PUT IT
TO WORK FOR US.
IT'S JUST A FASTER, SIMPLER WAY
OF CREATING THE URLs.
THIS IS CREATING THIS URL.
IF YOU LOOK CLOSELY YOU CAN SEE
THE SIMILARITIES.
THERE'S E-SEARCH COMMAND IS
SAYING WHICH UTILITY WE WANT TO
USE LIKE IN THE URL OVER HERE.
THIS IS SAYS TO USE PUBMED AS
THE DB PARAMETER AND THE H1N1
PARAMETER.
HERE'S OUR COMMAND AND HERE'S
THE URL AS FOR FETCH.
THE E-FETCH COMMAND TELLS US TO
USE THE E-FETCH UTILITY AS THE
PB TOLD US TO USE PUBMED.
WE HAVE ONE ARGUMENT THAT SAYS
WHICH WE WANT TO FETCH AND THE
FORMAT ARGUMENT THAT USES KML TO
GET XML RECORDS.
NOW, YOU WON'T ACTUALLY SEE THE
URLs IN PRACTICE, IT CREATES
THEM IN THE BACKGROUND AND QUERY
THE SERVER AND RETURNS THE
RESULTS WITHOUT LOOKING AT THE
URLs BUT IT DOESN'T GET US
FURTHER THAN BEFORE WHEN WE HAD
TO CREATE AN URL AND THEN TAKE
THE RESULTS OF THAT AND MANUALLY
FEED THEM IN THE NEXT URL.
WHERE E-DIRECT GETS GOOD IS TO
COMBINE MULTIPLE COMMANDS
TOGETHER.
THE TWO CHARACTERS TELL US TO
TAKE THE RESULT OF THE E-SEARCH
COMMAND AND TYPE THEM INTO OUR
NET COMMAND WHICH WE CAN FIND IN
THE NEXT LINE.
YOU'LL NOTICE IN THE E-FETCH
COMMAND THERE'S A FEW THINGS
MISSING.
THE COMMAND ON THE SECOND LINE
DOESN'T HAVE A DB ARGUMENT
BECAUSE IT ALREADY KNOWS WHICH
DATABASE WE'RE IN.
WE'RE IN PUBMED BUT IT DIDN'T IS
AN I.D. COMMAND.
THESE TWO LINES TOGETHER ALLOW
US TO SEARCH PUBMED AND PULL THE
COMPLETE XML RECORD OF ALL OF
OUR RESULTS AND NOW WE'RE READY
TO START ANSWERING SOME
QUESTIONS.
WE'RE GOING TO LOOK AT A FEW
EXAMPLES WE TALKED ABOUT AT THE
BEGINNING OF THE SESSION.
AGAIN, DON'T WORRY ABOUT
FOLLOWING ALONG WITH EVERY
DETAIL.
ALL THE CODE WE HAVE WE'RE
DEMONSTRATING TODAY IS AVAILABLE
ON OUR WEBSITE SO YOU CAN REVIEW
IT, TWEAK IT, REPURPOSE IT, DO
WHATEVER YOU WANT WITH IT.
LET'S START WITH THE FIRST
EXAMPLE.
REMEMBERING BACK TO THE
BEGINNING OF THE SESSION WE WERE
TRYING TO FIND AUTHORS PROLIFIC
IN THE LAST FEW YEARS IN OUR
CHOSEN TOPIC.
WE START WITH AN UN IX TERMINAL
AND I WRITE THEM OUT AND COPY
AND PASTE THE SCRIPT IN THE
TERMINAL LIKE SO.
AGAIN, I'M NOT GOING IN DETAIL
BUT YOU'LL SEE FAMILIAR
ELEMENTS.
WE HAVE AN E-SEARCH COMMAND AND
A SEARCH QUERY WITH A TOPIC
STREAM WITH A DATE RANGE.
WE TYPE IT IN OUR E-FETCH AND
THE RESULTS THAT OF THAT WE'LL
PIPE IN THE NEXT COMMAND WE
HAVEN'T TALKED ABOUT YET.
I'M NOT GOING TO GO INTO A LOT
OF DETAIL ABOUT EXTRACT RIGHT
NOW BUT EXTRACT IS AN CREDIBLY
USEFUL COMMAND BUILT INTO E
DIRECT.
IT EXTRACTS ELEMENTS FROM XML
OUTPUT INTO A TABLE.
IT'S VERY CUSTOMIZABLE.
YOU CAN SUES THE OUTPUT,
ARRANGEMENT, SEPARATORS,
EVERYTHING.
THIS MAKES E DIRECT USEFUL.
EXTRACT IS OUTSIDE OF THE SCOPE
OF TODAY'S CLASS BUT IF YOU TAKE
OUR E-DIRECT FOR PUBMED CLASS
WE'LL SPEND A LOT OF TIME
TALKING ABOUT EXTRACT.
WE'LL HAVE A FEW MORE LINES TO
ENSURE WE'RE GETTING THE DATA WE
WANT IN THE EXACT FORMAT AND
THEN WE HIT ENTER AND SEE THE
RESULTS.
NORMALLY THIS WOULD TAKE A FEW
MINUTES TO DO ALL THIS
PROCESSING BUT IT'S INSTANT.
WE GET OUR LIST OF A TOP TEN
AUTHORS WITH THE MOST ON THIS
TOPIC THE LAST COUPLE OF YEARS.
OUR NEXT EXAMPLE IT WILL VERY
SIMILAR, ACTUALLY.
STARTING WITH OUR BLANK COMMAND
PROMPT WE'LL SEE THE E-SEARCH
AND E-FETCH LINES ARE EXACTLY
THE SAME.
THE ONLY DIFFERENCE IS WHICH
ELEMENTS AREN'T EXTRACTING.
I RUN THE CODE INSTEAD OF
GETTING A LIST OF PROLIFIC
AUTHORS I GET A LIST OF THE TOP
TEN AGENCIES FREQUENTLY LISTED
AS FUNDING RESEARCH ON THESE
RECORDS.
NOW, FOR OUR LAST BIG DEMO WE'LL
SEARCH FOR THE H1N1 ASTHMA
SEARCH.
FOR THIS WE'LL DEMO IT LIVE SO
WHO KNOWS WHAT WILL HAPPEN.
BEAR WITH ME.
THIS IS THE COMMAND PROMPT AND
THIS IS THE CODE.
THE SERIES IS LONGER AND HAS
MORE LINES OF CODE TO PROCESS
THE DATA AND THEY'LL HELPS GET
THE DATA OUT IN THE CORRECT
FORMAT.
IT LOOKS LIKE NOTHING WHAT
HAPPENED WHICH IS ALWAYS A WORRY
WHEN YOU DO A LIVE DEMO HOWEVER
THE LAST BIT OF THE LAST LINE IS
IMPORTANT.
THE GREATER THAN SIGN THIS ONE
HERE IS GOING REDIRECT THE
RESULTS OF THIS SCRIPT INTO A
TEXT FILE.
WE CAN ON THAT IN A TEXT EDITOR
AND SEE THE RESULTS.
THERE YOU GO.
WE HAVE A NUMBER OF THE SEARCH
RESULTS PUBLISHED IN EACH OF
THOSE YEARS.
WE CAN ALSO OPEN UP THE FILE TO
ANALYZE OR VISUALIZE THE DATA
LIKE MICROSOFT XCEL.
NOW WE HAVE DATA WE JUST GOT OUT
OF PUBMED UP EXCEL WHERE WE CAN
ANALYZE IT OR VISUALIZE IT AND
USE IT AS THE BASIS FOR THE
CHART THAT LETS US RECREATE THE
HISTOGRAM.
I HAVE CREATED IT AHEAD OF TIME
AND IT LOOKS LIKE THAT BASED ON
THE SAME DATA.
THIS IS A DIFFERENT EXAMPLE.
.
YOU CAN EXTRACT A SEARCH FOR USE
DIFFERENT DATA ELEMENTS AND
VISUALIZE IT DIFFERENTLY.
BECAUSE WE'RE USING E UTILITY WE
HAVE A MORE FLEXIBLE TOOL.
I WANT TO ZOOM BACK OUT TO THE
BIG PICTURE.
THE WHOLE POINT IS FIGURING OUT
HOW TO ACCESS DIFFERENT PUBMED
DATA.
AS WE'VE SEEN E-DIRECT MAKES
USING E-UTILITIES EASIER IF
YOU'RE USING IN AN UNIX TERMINAL
ENVIRONMENT.
HOWEVER, IF YOU'RE WORKING IN A
DIFFERENT ENVIRONMENT YOU'LL
PROBABLY WANT TO USE DIFFERENT
TOOLS.
YOU MAY BE FAMILIAR WITH
STATISTIC ANALYSIS AND SAY YOU
WANT TO DO ANALYSIS OR
VISUALIZATION OF PUBMED DATA AND
WANT TO USE R AND TO GET AT THE
DATA YOU'LL WANT TO USE THE SAME
E-UTILITIES API BUT BECAUSE
YOU'RE IN A DIFFERENT
ENVIRONMENT YOU'LL WANT
DIFFERENT RULES.
REMEMBER IT ONLY WORKS IN UNIX.
THERE'S PACKAGES OF COMMANDS
DESIGNED TO WORK IN THE
ENVIRONMENT.
THESE ARE LIKE R-EUTIL MAKE IT
EASIER TO ACCESS PUBMED DATA IN
THE R ENVIRONMENT.
NOW, IF YOU'RE WORKING IN A
DIFFERENT ENVIRONMENT BUT HAVE
DIFFERENT TOOLS ALTOGETHER MAYBE
YOU EVEN HAVE TO BUILD YOUR OWN
TOOLS BUT STILL USE THE API.
IN A WEB BROWSER YOU MAY NOT USE
SPECIAL TOOLS AT ALL.
WE CAN ACCESS THEM DIRECTLY AND
THIS MAKES IT FLEXIBLE.
YOU CAN ACCESS PUBMED DATA NO
MATTER THE ENVIRONMENT YOU'RE
IN.
YOU JUST HAVE TO FIND THE RIGHT
TOOLS.
HOPEFULLY THIS INTRODUCED YOU TO
THE IDEA AND THIS IS JUST THE
BEGINNING.
YOU'RE INTERESTED IN SOMETHING
MORE INTERACTIVE.
WE HAVE A HANDS-ON WORKSHOP TO
INTRODUCE YOU TO THE GET DATA
YOU WANT IT'S A FIVE-PART SERIES
STARTING JULY 10.
YOU CAN SIGN ON AND KATE PUT A
DIRECT LINK IN THE CLASS PAGE IN
THE CHAT BOX.
WE HOPE TO DEVELOP FURTHER
COURSES BASED ON WHAT WE HEAR
FROM YOU SO LET US KNOW BASED ON
THE EVALUATION FORM AT THE END
OF THE SESSION WHAT ELSE YOU'D
LIKE TO SEE.
OR IF YOU'LL BE AT THE MLA
CONFERENCE AT IN SEATTLE STOP BY
AND SAY HI.
WE'LL BE THERE AND WOULD LOVE TO
HEAR FROM YOU DIRECTLY ON WHAT
YOU THINK ABOUT THE PROGRAM AND
WHERE YOU'D LIKE IT TO GO.
IF YOU ARE INTERESTED IN
CONTINUING TO LEARN CHECK OUT
THE INSIDERS GUIDE WEBSITE I
MENTIONED A COUPLE TIMES AT
DATAGUIDE.NLM.gov.
WE HAVE RECORDINGS AND SLIDES OF
PREVIOUS CLASSES AND RECORDINGS
FOR THIS CLASS WILL BE UP THERE
AND WE HAVE DETAILED
INSTRUCTIONS FOR E-DIRECT.
AND IF YOU HAVE QUESTIONS ABOUT
ANY MATERIAL PRESENTED OR ANY
OUR CLASSES USE THAT LINK TO GET
IN TOUCH WITH US.
YOU CAN ALSO VIEW THE LINKS AND
WHILE WE FOCUS ON THE TOOLS TO
ACCESS PUBMED DATA NCBI COVERS
ALL THE E-UTILITIES IT WORKS
WITH.
BEFORE WE FINISH UP I HAVE ONE
MORE EXAMPLE I WANT YOU TO THINK
ABOUT.
SAY WE WANT TO KNOW ABOUT THE
PORTFOLIO AND OUR ORGANIZATION
HAS MANY AUTHORS AND RESEARCH
COMPONENT WITH A VARIETY OF
NAMES, ACRONYMS, ABBREVIATIONS,
ETCETERA ALL OF WHICH CAN BE
FOUND IN THE AFFILIATION OF AN
ARTICLE.
HOW CAN WE EFFICIENTLY SEARCH
FOR OUR ORGANIZATION AND ANALYZE
THE TOPICS OF THE PUBLISHED
ARTICLE.
THIS IS THE KIND OF QUESTION WE
GET AT CUSTOMER SERVICE A LOT
AND IT IS DOABLE WITH E
UTILITIES.
I'M NOT GOING TO DEMO IT NOW
BECAUSE WE DON'T HAVE ENOUGH
TIME BUT WE HAVE SAMPLE CODE ON
THE WEBSITE YOU CAN CHECK OUT
AND IF IT'S A KIND OF QUESTION
YOU HAVE OR FOLKS IN YOUR
INSTITUTION HAVE LOOK AT OUR
SAMPLE CODE AND SEE IF IT'S
SOMETHING YOU CAN ADAPT.
EVEN IF IT'S NOT RELEVANT TO YOU
CONSIDER LOOKING AT THE CODE
ANYWAY AS A PRACTICE EXERCISE.
START TO THINK ABOUT HOW YOU CAN
ACCOMPLISH THIS OR ADAPT IT TO
SOMETHING RELEVANT IT YOU.
IT MAY BE SIMILAR TO A PROBLEM
YOU'D HAD SOLVING IN THE PAST
BECAUSE YOU DIDN'T HAVE IT RIGHT
TOOLS FOR THE JOB.
I'M SURE MANY OF YOU HAVE JOBS
UTILITIES ARE USED FOR SO GET
OUT THERE AND OUT E-UTILITIES TO
WORK.