HELLO AND THANK YOU FOR JOINING US FOR E-DIRECT
FOR PUBMED THIS IS THE FIRST PART OF THE INSIDER'S
GUIDE TO ACCESSING NLM DATA.
MY NAME IS PETER SEIBERT AND I AM HERE WITH SARAH
HELSON WHO WILL BE PRESENTING THIS
THURSDAY'S SESSION AS WELL AS MIKE DAVIDSON WHO
WILL BE ANSWERING YOUR QUESTIONS IN THE CHAT BOX.
OVER THE COURSE OF THE FIVE SESSIONS WE'LL
INTRODUCE YOU TO USE E-DIRECT TO SEARCH FOR,
RETRIEVE AND MANIPULATE PUBMED DATA IN AN UNIX
ENVIRONMENT AND THERE'S A LOT TO COVER BUT WE'LL
GO OVER THE BASICS AND GIVE YOU ENOUGH TO GET
STARTED TO CONTINUE WITH THE RESOURCES TO CONTINUE
EXPLORING AND LEARNING ON YOUR OWN.
DURING THIS CLASS, IF YOU HAVE QUESTIONS FEEL FREE
TO POST THOSE IN THE CHAT BOX.
PLEASE MAKE SURE THAT YOU'VE SELECTED TO SEND
THAT TO EITHER ALL PARTICIPANTS SO THAT OTHER
PEOPLE WHO ARE IN THIS CLASS -- THEY MAY HAVE THE
SAME QUESTIONS YOU HAVE SO IT WILL BE GOOD FOR
THEM TO SEE.
IF YOU DON'T WANT TO ADDRESS IT TO EVERYONE
YOU CAN SELECT ALL PANELISTS AND THAT WILL
COME TO BOTH SARAH AND MIKE WHO WILL BE ABLE TO TAKE
CARE OF YOUR QUESTIONS AS WE GO IF IT'S A QUESTION
THAT'S PERTINENT TO THE WHOLE GROUP YOU MAY SAVE
SOME THEM UNTIL THE END AND ADDRESS IT AS PART
OF THE CLASS AT THE END.
SO GO AHEAD.
AS WE'RE GOING, IF YOU HAVE QUESTIONS, POP THEM
INTO THE CHAT BOX.
THIS COURSE IS MADE UP OF FIVE SESSIONS AND TODAY
WE'RE GOING TO TALK ABOUT HOW WE USE E-DIRECT
TO GET DATA FROM PUBMED, INCLUDING HOW TO DOWNLOAD
THE FULL XML PUBMED RECORD.
NEXT TIME WE'LL START TALKING ABOUT HOW TO EXTRACT
SPECIFIC DATA ELEMENTS FROM XML RECORDS AND RANGE THEM
INTO A CUSTOM OUTPUT.
THAT'S THIS THURSDAY WITH SARAH AND NEXT WEEK
AT THE SAME TIME, NEXT MONDAY, WE'LL GET
INTO MORE SPECIFIC WAYS TO
FORMAT AND CUSTOMIZE RESULTS AND LOOK AT SOME
UNIX TOOLS YOU CAN USE WHEN BUILDING
YOUR SOLUTIONS.
IN PART FOUR WE'LL EXPLORE THE XTRACT
CONDITIONAL ARGUMENTS WHICH LETS YOU FILTER YOUR OUTPUT.
AND FINALLY, IN TWO WEEKS WE'LL PUT ALL OF THE PIECES
TOGETHER AND LOOK AT SOME STRATEGIES FOR DEVELOPING
AND BUILDING SOLUTIONS TO REAL-WORLD PROBLEMS.
I KNOW MOST OF YOU ALREADY HAVE EXPERIENCE
SEARCHING IN PUBMED SO IT'S IMPORTANT TO
UNDERSTAND THE MANY OF THE TASKS AND CONCEPTS
AREN'T NECESSARILY COMPLETELY NEW TO YOU.
FOR EXAMPLE, YOU KNOW HOW TO SEARCH PUBMED AND
RETRIEVE THE RECORD AND REVIEW IT IN DIFFERENT
FORMATS BUT TODAY WE'LL TEACH YOU NEW TECHNIQUES
TO ACCOMPLISH THE TASKS IN A PROGRAMMATIC WAY.
THIS IS GOING TO LAY THE FOUNDATION FOR THE REST
OF THE COURSE WHERE WE'LL BUILD ON THESE NEW TECHNIQUES
AND SHOW YOU HOW TO EXTRACT THE SPECIFIC
DATA YOU'RE LOOKING FOR IN CUSTOM OUTPUT
FORMATS WHICH IS SOMETHING YOU CAN'T DO
EASILY IN PUBMED.
I EXPECT MOST OF YOU HAVE TAKEN THE WELCOME TO
UTILITIES FOR PUBMED CLASS WE OFFERED WEEKS AGO OR
REVIEWED THE VIDEO WE HAVE POSTED.
BUT I'M GOING TO START WITH A BRIEF RECAP SO WE'RE
ALL ON THE SAME PAGE AND JUST TO REFRESH
EVERYONE'S MEMORY ABOUT WHAT E-UTILITIES CAN DO,
WE'RE GOING TO BRIEFLY TALK ABOUT UNIX WHICH IS
THE ENVIRONMENT WE'LL BE WORKING IN THROUGHOUT
THE COURSE.
I'LL SHOW YOU HOW TO CONSTRUCT A PUBMED
SEARCH USING THE ESEARCH COMMAND AND HOW TO VIEW
THE PUBMED TRANSLATION OF THAT SEARCH SIMILAR TO
WHAT YOU CAN DO IN THE SEARCH DETAILS IN THE
WEB VERSION OF PUBMED.
WE'LL TALK ABOUT DOWNLOADING CITATION
DATA FROM PUBMED RECORD WITH E-FETCH AND I'LL FINISH
UP BY TALKING ABOUT SCRIPTS AND HOW TO COMBINE
A SERIES OF COMMANDS INTO A DATA PIPELINE.
BEFORE I RECAP THE CONTENTS OF WELCOME TO
UTILITIES FOR PUBMED I WANT TO REMIND YOU OF A
GENERAL THEME IN THE CLASS THAT WILL BE
OVERARCHING IN THE NEXT FEW WEEKS OF CLASS.
E-UTILITIES AND SPECIFICALLY E-DIRECT
ARE BEST UTILIZED TO GET YOU THE DATA YOU NEED IN
EXACTLY THE FORMAT YOU WANT TO USE IT IN.
WE'LL COME BACK TO THAT A LITTLE BIT LATER ON BUT
FIRST LET'S GO OVER THE WELCOME TO E-UTILITIES CLASS
AND I WILL PROVIDE YOU A FEW DEFINITIONS TO START WITH.
AN API STANDS FOR APPLICATION PROGRAM
INTERFACE AND IS A SET OF TOOLS FOR PROTOCOLS FOR
BUILDING SOFTWARE APPLICATION.
SO BULDING ON THAT GENERAL DEFINITION, THE
NLM API SUITE, E-UTILITIES, IS A SET OF TOOLS, ROUTINES
AND PROTOCOLS THAT ALLOW TO YOU INTERACT DIRECTLY
WITH THE DATA IN OVER TWO DOZEN NCBI DATABASES AND
PUBMED AND MESH AND PMC.
WE HAVE ALL THE DATABASES LISTED IN OUR
DOCUMENTATION IF YOU'D LIKE TO TAKE A LOOK AND
SEE WHICH OTHER ONES ARE SUPPORTED BY E-UTILITIES.
SO THE E-UTILITIES API IS JUST A SET OF RULES FOR
QUERYING THE BACK END NLM DATABASES AND THOSE
QUERIES ARE IN THE FORM FORM OF A STANDARD URL.
AND THE RESPONSE YOU GET TO THE QUERY DEPENDS ON
HOW YOU BUILD THE URL.
YOU CHOOSE ONE OF THE NINE UTILITIES TO SPECIFY
WHAT TYPE OF QUERY YOU'RE ASKING AND SELECT
PARAMETERS TO FIND THE DETAILS OF THESE QUERY.
IDEALLY, YOU DON'T WANT TO CREATE THE URLs BY HAND.
WITH THE TIME SPENT YOU MIGHT JUST BE BETTER OFF
GOING INTO PUBMED AND BUILDING YOUR OWN
SEARCHES FOR DATA.
WHAT YOU REALLY WANT TO DO IS USE E-UTILITIES IN A
PROGRAMING ENVIRONMENT TO COMBINE MUTIPLE QUERIES
IN SEQUENCE, USING THE RESULTS FROM ONE TO
CREATE THE NEXT.
THE PROGRAMMING ENVIRONMENT ALLOWS YOU TO --
ALSO GIVES YOU MORE OPTIONS FOR
MANIPULATING THE DATA RATHER THAN TAKING THE
XML OR TEXT WHICH IS WHY NCBI DEVELOPED E-DIRECT.
IT'S A SET OF TOOLS WITH THE E-UTILITIES URL
CREATION RULES BUILT IN TO IT AND INCLUDES A
POWERFUL TOOL FOR EXTRACTING OR PARSING
SPECIFIC DATA ELEMENTS FROM XML AND WRITTEN TO
WORK IN AN UNIX ENVIRONMENT.
SO I SAID AT THE BEGINNING OF THE CLASS
THAT THIS WAS GOING TO BE INTERACTIVE AND THERE
WOULD BE SOME POINTS FOR YOU TO EITHER ASK
QUESTIONS OR DO EXERCISES AND I HAVE A QUESTION FOR
YOU AND I'D LIKE TO TEST THIS PARTICIPANTS WINDOW
SO IF YOU HAVE USED UNIX AND SAY YOU WERE A
DATABASE ADMINISTRATOR OR SERVER ADMINISTRATOR JUST
LEARNED UNIX A WHILE AGO, SO IN THE PARTICIPANTS WINDOW,
PLEASE HIT THE GREEN CHAT BOX AND THAT WILL LET ME
KNOW IF YOU HAVE EXPERIENCE IN UNIX
OR NOT.
I'LL GIVE IT A COUPLE MINUTES TO TRY THAT OUT.
THANKS, CINDY.
HOPEFULLY EVERYONE IS FINDING THIS.
IT'S IN THE PARTICIPANTS WINDOW AND RIGHT
AT THE BOTTOM NEAR THE LIST OF NAMES.
THANKS, IT'S HELPFUL TO KNOW.
ALMOST ALL THE CLASS YOU'LL BE WORKING IN
UNIX AND E-DIRECT.
AND UNIX IS AN OPERATING SYSTEM THAT WILL ALLOW
YOU TO INTERACT DIRECTLY WITH YOUR COMPUTER VIA A
COMMAND LINE INTERFACE WHICH IS BASICALLY
A TEXT PROMPT.
I DON'T KNOW IF SOME OF YOU WERE ONLINE WHEN I
FIRST GOT ON BUT I HAD MY CYGWIN E-DIRECT TERMINAL
WINDOW OPEN AND JUST PUT IT ON MY SCREEN SO YOU
WILL SEE MY USER NAME AND SEE MY COMMAND LINE WITH
A PROMPT.
THAT'S ALL IT IS.
THAT'S ALL WE'RE TALKING ABOUT HERE.
YOU TYPE INSTRUCTIONS TO THE COMPUTER IN UNIX AND
TELL THE COMPUTER TO EXCECUTE THE INSTRUCTIONS
AND WAIT FOR THE RESULT AND YOU MAY SEE A COMMAND
LINE INTERFACE REFERRED TO AS A SHELL
OR A TERMINAL.
I MAY OR MAY NOT REFER TO EITHER OF THESE
TWO INTERCHANGEABLY.
THEY'RE NOT SYNONOMOUS BUT FOR ALL INTENTS AND
PURPOSES FOR TODAY'S CLASS WE'LL USE THEM
INTERCHANGEABLY AND FOR THE COURSE.
DEPENDING ON WHAT TYPE OF COMPUTER YOU'RE USING,
YOU MAY HAVE OPTIONS FOR WHICH TERMINAL YOU WANT
TO USE.
I'M ON A WINDOWS PC SO I DOWNLOADED THE EMULATOR
CYGWIN WHICH IS IN OUR INSTRUCTIONS FOR DOWNLOAD
WHICH YOU'LL SEE.
IF YOU'RE USING A MAC OR LINUX THE TERMINAL
YOU'RE USING IS PROBABLY DIFFERENT.
REGARDLESS OF THE NATIVE OPERATING SYSTEM YOU ARE
USING, IT ALL BOILS DOWN TO THE SAME THING, YOU
GET A PROMPT TO TYPE INSTRUCTIONS TO THE
COMPUTER, EXCUTE THEM AND WAIT AND SEE THE RESULTS.
LIKE MOST OPERATING SYSTEMS UNIX IS MEANT TO
WORK WITH FILES AND CREATE, DELETE
OR EDIT THEM.
MANY WON'T BE NEEDED TODAY BUT THEY'LL HELP
YOU UNDERSTAND WHY THINGS DID OR DID NOT WORK.
WE'LL BE USING SOME OF THE FILE MANIPULATION
TOOLS IN SOME OF THE LATER SESSIONS AND I'LL POINT
THEM OUT AS WE GET TO THEM TODAY.
THE MODULAR DESIGN MEANS EACH PROGRAM IS SUPPOSED
TO DO ONE THING BUT VERY WELL.
DEVELOPERS AND PROGRAM -- PROGRAMMERS HAVE THEIR
OWN PERSONAL TASTE AND DEVELOPED DIFFERENT
TOOLS TO DO ALMOST THE SAME THING BUT
SLIGHTLY DIFFERENTLY.
THIS MEANS THERE'S OFTEN MULTIPLE WAYS TO DO THE
SAME THING AND THEREFORE DIFFERENT SOLUTIONS
THE SAME PROBLEM.
YOU'LL SEE THIS TODAY.
BECAUSE EACH PROGRAM GENERALLY DOES ONLY ONE
THING AS I PREVIOUSLY SAID YOU WILL OFTEN WANT
TO COMBINE A SERIES OF PROGRAMS TOGETHER IN
SEQUENCE USING THE OUTPUT OF ONE AS THE
INPUT FOR THE NEXT.
UNIX IS BUILT AROUND THIS IDEA AND FACILITATES THE USE
OF SCRIPTS WHICH ARE MINI PROGRAMS THAT CAN COMBINE
MULTIPLE PROGRAMS TOGETHER.
SO I'M GOING DEFINE A COUPLE TERMS WE'RE GOING
TO USE TODAY THROUGHOUT THE CLASS AS WE'RE
WORKING THROUGH EXERCISES IN E-DIRECT.
COMMANDS ARE INSTRUCTIONS GIVEN BY A USER TELLING THE
COMPUTER TO DO SOMETHING LIKE RUN
A PROGRAM.
THAT IS ONE OF THE COMMANDS YOU'RE
MOST USED TO.
ARGUMENTS CAN PROVIDE DATA TO BE USED AS THE
COMMANDS INPUT OR MODIFY THE BEHAVIOR
OF THE COMMAND.
OUR EXAMPLE ON THE SLIDE, YOU SEE E-SEARCH
IS OUR COMMAND.
AND I WILL REITERATE THIS MANY TIMES THROUGHOUT THE CLASS.
THIS DASH DB PUBMED IS THE ARGUMENT.
IT PROVIDES INPUT TO ESEARCH, TELLING THE COMMAND
THAT WE'LL BE USING PUBMED AS OUR DATABASE.
IN OUR SECOND EXAMPLE, E-INFO IS OUR COMMAND, WITH THIS
DBS ARGUMENT RATHER THAN SUPPLYING DATA MODIFYING
THE BEHAVIOR OF E-INFO, TELLING IT THAT WE WANT
TO USE THE DBS OPTION WHEN WE EXECUTE THE COMMAND.
AS YOU CAN SEE HERE, DIFFERENT COMMANDS HAVE
DIFFERENT ARGUMENTS.
SOME HAVE REQUIRED ARGUMENTS, SOME ARE OPTIONAL,
AND SOME COMMANDS DON'T ACCEPT ANY ARGUMENTS AT ALL.
WE OFTEN WANT TO TAKE THE OUTPUT FROM ONE COMMAND AND
USE IT AS AN INPUT FOR ANOTHER COMMAND.
RATHER THAN RUNNING COMMAND THAT SAVES
ITS OUTPUT TO A FILE, THEN RUNNING A SECOND COMMAND USING
THAT FILE AS AN ARGUMENT, WE CAN COMBINE OR PIPE
THESE TOGETHER USING A UNIX FUNCTION
THE PIPE CHARACTER IS JUST OVER THE ENTER KEY, YOU CAN GET TO IT BY SHIFT-BACKSLASH
IT PIPES OR CHANNELS THE OUTPUT OF ONE
COMMAND INTO THE NEXT.
WE'LL LOOK AT THIS IN ACTION TOWARDS THE END
OF TODAY'S CLASS, AND WE'LL HAVE AN EXERCISE USING THAT COMMAND.
WHEN WORKING IN ANY PROGRAMMING LANGUAGE, ONE THING
YOU SHOULD BE PREPARED FOR IS FAILURE
AND THE NEED FOR REPETITION AND TROUBLESHOOTING.
SOMETHING YOU'RE TRYING TO DO IS NOT GOING TO WORK,
AND YOU MIGHT HAVE TROUBLE FIGURING OUT WHY IT FAILED.
WITH UNIX, THE DETAILS MATTER.
IT IS CASE SENSITIVE AND SPACE SENSITIVE.
IF YOU DON'T DO EVERYTHING IN THE CORRECT SYNTAX
OR ADD OR DROP A SPACE, YOUR CAREFULLY THOUGHT OUT SCRIPT
WON'T WORK.
A LOT OF TIMES IT WON'T TELL YOU WHAT
YOU DID WRONG.
THE BEST ADVICE IS TO HAVE PATIENCE AND BE WILLING
TO EXPERIMENT.
I CAN TELL YOU RIGHT NOW, THIS IS A LIVE CLASS AND
I WILL BE DOING LIVE DEMOS.
SO YOU ARE MOST LIKELY GOING TO SEE THIS IN ACTION TODAY.
A COUPLE OF TIPS, TEST EARLY, OFTEN, AND WITH INCREMENTAL CHANGES.
WHEN YOU ARE RUNNING A SCRIPT AND IT FAILS,
IT CAN BE HARD TO FIGURE OUT WHICH LINE IS THE ROOT CAUSE.
AS I SAID, IT IS SPACE SENSITIVE IT'S OFTEN HARD TO LOOK THROUGH TEN LINES OF DATA -- A SCRIPT
AND SEE WHERE YOU MISSED A SPACE.
TRY EACH COMMAND SEPARATELY TO IDENTIFY THE
FAILURE POINTS.
IF YOU CAN, USE SMALL SETS OF DUMMY DATA: OFTEN JUST A SMALL SET OF PMIDS
WHERE YOU MAY BE FAMILIAR WITH CITATION AND METADATA
BEHIND THEM.
YOU CAN ADD IN A SEARCH STRATEGY THAT IS ARTIFICIALLY
LIMITED BY DATE TO SHRINK YOUR RETRIEVAL SET.
THIS WILL ALLOW YOU TEST FASTER AND MORE FREQUENTLY.
AND KNOW WHEN YOU NEED HELP.
THAT'S ALWAYS A GREAT PIECE OF ADVICE.
THE INTERNET IS A GREAT PLACE TO START.
THERE ARE GREAT ESTABLISHED UNIX USER COMMUNITIES OUT
THERE WITH TERRIFIC DOCUMENTATION AND ACTIVE USERS
USERS WHO ACTIVELY ANSWER QUESTIONS FOR YOU.
JUST GO TO A WEB BROWSER AND SEARCH FOR THE WORD UNIX
AND EXACTLY WHAT YOU'RE TRYING TO DO AND MOST LIKELY
YOU'LL FIND SUPPORTING DOCUMENTATION
FOR THAT.
FOR MY UNIX EMULATOR, I'M USING CYGWIN.
AS I KNOW, QUITE A FEW OF YOU ARE OUT THERE.
SOME OF THE KEYS ARE A LITTLE BIT DIFFERENT.
CONTROL C IS NOT GOING TO COPY.
ONLY CONTROL INSERT AND THE INVERSE WOULD BE
SHIFT INSERT FOR PASTING.
YOU CAN ADJUST THE OPTIONS IN CYGWIN WHICH IS HELPFUL.
A TIP FOR ALL USERS: CONTROL + C CANCELS.
THAT'S A GREAT WAY TO GET OUT OF MISTAKES.
I HAD TO USE IT EARLIER TODAY WHEN I WAS DEMOING A COUPLE SCRIPTS FOR THIS CLASS.
UP AND DOWN ARROW KEYS CYCLE THROUGH YOUR HISTORY COMMANDS.
YOU'RE GOING TO SEE ME DOING THIS A LOT TODAY
THROUGH THE CLASS.
SO WHEN YOU SEE A SCRIPT POP UP ON MY
EDIRECT SCREEN, MOST LIKELY I'M LIKELY CYCLING
THROUGH OLDER ONES THAT I'VE ALREADY PUT IN THERE.
AND CLEAR -- TYPING "CLEAR" CLEARS YOUR SCREEN.
KEEPS ALL OF YOUR HISTORY IN THERE BUT GIVES YOU A BLANK
PALLET TO WORK OFF OF.
SO I WENT OVER QUITE A BIT WITH THAT RECAP.
I'LL PAUSE TO SEE IF ANYONE HAS QUESTIONS.
GO AHEAD AND PUT THEM IN THE CHAT.
SO WE'RE GOING TO BEGIN IF YOU REMEMBER FROM THE AGENDA
WITH E-SEARCH.
DOES ANYONE KNOW FROM THE WELCOME OR JUST WANT TO
TAKE A WILD GUESS AT WHAT E-SEARCH DOES?
YOU CAN PUT THAT IN THE CHAT.
AGAIN, ONE OF OUR FIRST COMMANDS THAT WE'RE GOING TO BE WORKING ON HERE.
OKAY, I SEE ONE: "SEARCHES PUBMED." I THINK I LIKE NINA'S: "SEARCHES VIA EDIRECT."
I THINK THAT'S A FAIR STATEMENT THAT IT
SEARCHS LIKE PUBMED.
SO E-SEARCH CONDUCTS A SEARCH OF A SPECIFIED DATABASE
WHICH IS WHY I KIND OF LIKE SAYING EDIRECT MORE THAN JUST SPECIFICALLY PUBMED
AND RETRIEVES THE UNIQUE IDENTIFIER OF RECORDS
THAT MEETS YOUR SEARCH CRITERIA.
FOR PUBMED THAT'S WHAT WE'RE TEACHING.
THE UNIQUE IDENTIFIERS WOULD BE THE PMIDS OF CITATIONS
THAT MATCH OUR QUERY. SO THOSE PMIDS
ARE UNIQUE IDENTIFIERS FOR EACH RECORD WITHIN PUBMED
THAT WE'RE RETRIEVING.
SO WE'RE GOING TO START WITH THE BASIC COMMAND
THAT ALLOWS YOU TO SEARCH OUR DATATBASE: PUBMED.
AND I'LL BE HOPPING BACK AND FORTH FROM MY SLIDES
TO PUBMED TO MY E-DIRECT WINDOW.
SO EVERYONE SHOULD SEE MY COMMAND PROMPT.
AS I SAID, I'M GOING TO START WITH JUST A BASIC
SEARCH IN PUBMED AND LOOKING FOR CITATION
HAVING TO DO WITH SEASONAL AFFECTIVE DISORDER.
SO I'LL BEGIN WITH MY COMMAND WHICH AGAIN
IS E-SEARCH.
THEN MY FIRST ARGUMENT I NEED TO SPECIFY THE
DATABASES AND WE'LL BE WORKING IN PUBMED
THROUGHOUT THE COURSE.
THE NEXT ARGUMENT I HAVE WHICH IS GOING TO AFFECT
THE COMMAND LINE AND WHAT IT IS JOINED IS WE
WANT TO QUERY THAT DATABASE.
AS I SAID, I'M LOOKING FOR SEASONAL AFFECTIVE
DISORDER LITERATURE.
YOU CAN SEE I ENCASED THAT QUERY IN QUOTES AND THEN SIMPLY
HIT ENTER.
YOU CAN SEE MY FIRST MISTAKE.
REMEMBER, I TOLD YOU NOT ONLY IS EDIRECT VERY
SPECIFIC BUT PUBMED IS TOO.
SO THE FIRST THING
YOU'RE GOING TO -- SO NOW THE FIRST THING
YOU'LL SEE ON THE SCREENIS THE SNIPPET INCLUDING
A COUPLE ITEMS.
SOME WE'LL DISCUSS NOW AND SOME I'LL GET TO IN
A MOMENT.
FIRST THING I WANT TO BRING YOUR ATTENTION TO
IS MIDWAY THROUGH YOU SEE THIS COUNT. THE COUNT IS TELLING US HOW MANY PMIDS WE'VE
RETRIEVED THAT MET THE CRITERIA OF SEASONAL
AFFECTIVE DISORDER.
I SAID THAT HOW E-DIRECT WORKS AND THIS E-SEARCH
WORKS IS WE'RE QUERYING THE BACK END OF THE
PUBMED DATABASE.
TO DEMONSTRATE THAT I'LL BRING OVER PUBMED AND
I'M GOING TO DO THE EXACT SAME SEARCH IN PUBMED.
SO I DON'T MISTYPE AGAIN LET ME USE THE PULL DOWN.
YOU'LL SEE 1,695 CITATIONS RETRIEVED IN
PUBMED MATCHING WHAT WE GOT OUT OF E-DIRECT.
I WANT TO DRAW YOUR ATTENTION TO ONE OTHER
THING WHILE WE'RE HERE IN PUBMED.
I'M GOING TO SCROLL DOWN ON THE RIGHT WE GO TO
THE DISCOVERY WINDOW.
YOU'LL SEE THIS SEARCH DETAILS. I'M GOING TO HIT SEE MORE SO EVERYONE CAN READ THIS.
I WANT TO GO OVER THIS A LITTLE BIT.
SO I ENTERED SEASONAL AFFECTIVE DISORDER AS
MY SEARCH TERMS IN PUBMED.
AND I OPENED THIS QUERY TRANSLATION AND THIS
IS SHOWING YOU WHAT PUBMED SEARCHED FOR.
THIS IS A PROCESS CALLED AUTOMATIC TERM MAPPING.
WHAT WE'VE DONE IS TAKEN THIS ENTIRE QUERY PHRASE AND MAPPED
DIRECTLY TO A MESH TERM FOR SEASONAL EFFECTIVE
DISORDER AND TO MAKE SURE WE'RE GETTING THE BEST
RETRIEVAL POSSIBLE WE BREAK IT APART AND TAKE
YOUR ENTERED TERMS AND SEARCH FOR THEM EACH IN
ALL FIELDS.
THEN WE KEEP THE PHRASE TOGETHER AND SEARCH FOR
IT IN ALL FIELDS.
SO THIS IS VERY VERY HELPFUL IN REFINING YOUR
SEARCH AND SOMETHING YOU'LL WANT TO BE ABLE
TO DO IN E-DIRECT AS WELL.
I'M GOING TO GO BACK TO MY EDIRECT WINDOW
I TOLD YOU I WOULD CLEAR MY SCREEN OFTEN.
AGAIN, ALL THAT DID IS CLEAR MY SCREEN AND I'M
USING MY UP AND DOWN ARROWS TO BRING BACK THAT QUERY WE
JUST DID.
THE SEARCH IN PUBMED FOR SEASONAL EFFECTIVE
DISORDER.
I'M GOING TO AMEND THIS WITH THE ARGUMENT-LOG
AND WE'VE RETRIEVED WHAT LOOKS LIKE THAT SAME XML
SNIPPET AND AT THE END YOU'LL SEE THIS TEXT AND
THIS A REPRESENTATION OF THE SEARCH DETAILS.
WE ENTERED THE PHRASE AND IT'S INDEED A MESH TERM
AND WE BROKE IT APART AND COMPLETED THE
SEARCH.
ONE OTHER ITEM YOU'RE GOING TO SEE
IN THE XML IS THE WEB ENVIRONMENT AND THE QUERY
KEY.
I'M REALLY NOT GOING GET IN THIS TODAY.
WE'LL TALK ABOUT IT IN OTHER CLASSES BUT RIGHT
NOW I WANT YOU TO KNOW THIS IS THE ITEM THAT
WILL ALLOW YOU TO STRING MULTIPLE COMMANDS
TOGETHER AND USE THE RESULTS OF ONE TO CREATE
COMMANDS AND PROCESSES.
ANOTHER THING I WANTED TO POINT OUT IS THERE'S A
COUNT OF PMIDs BUT THERE'S NOT ACTUALLY
PMIDs LISTED AND THERE'S A REASON FOR THIS.
IT HAD TO DO WITH, HOW
WE'RE TALKING ABOUT TESTING AND GETTING YOUR
RESULTS OUT.
IF YOUR RESULT SET IS HUGE YOU CAN HAVE HUNDREDS OF
THOUSAND OF PMID'S SCROLLING DOWN THE PAGE.
THIS SHOWS YOU THE SIZE OF YOUR RESULTS SET AND LETS YOU DECIDE WHAT
YOU WANT TO DO WITH IT, WHICH DIRECTLY TIES BACK INTO WHAT YOU'LL SEE IS A RECURRING THEME OF OURS:
BUILD SLOWLY AND ONE STEP AT A TIME, SO YOU CAN UNDERSTAND THE DATA THAT YOU'RE RETRIEVING
REMEMBER, WE CAN DO ALL THESE SAME SEARCHS WE DO
IN THE WEB VERSION OF PUBMED IN E-SEARCH.
LET ME GO BACK INTO PUBMED.
SO IN PUBMED, ONE WAY TO REFINE AND CLARIFY YOUR
SEARCH IS TO ADD BOOLEAN OPERATORS.
OR WE CAN TAG TERMS TO RESTRICT OUR SEARCH TO A
CERTAIN FIELD.
FOR EXAMPLE, IF I WANTED TO RETRIEVE ALL THE
ARTICLES IN PUBMED ON THE SUBJECT OF MALARIA BUT I
WANTED TO LIMIT THAT TO A JOURNAL I HAVE A
SUBSCRIPTION TO I'D GO INTO THE SEARCH IN PUBMED
AND TYPE MY SEARCH TERM, IN OUR CASE IS MALARIA,
AND I WOULD JOIN THAT WITH "AND" ALL
CAPITALIZED.
AND THEN I'LL SPECIFY THE JOURNAL THAT I HAVE A SUBSCRIPTION TO
AND FOR MY CASE I HAVE SUBSCRIPTION TO JOURNAL TO RESTRICT
JAMA. I'M GOING TO TAG THAT WITH JOURNAL, TO RESTRICT IT ONLY FROM THAT JOURNAL.
TO ARTICLES FROM THAT JOURNAL.
I'LL COME BACK TO MY SEARCH DETAILS.
I WANT TO SHOW YOU THAT
WAS A MESH TERM, SO WE GOT AN ALL FIELDS
AND MESH
AND THEN JAMA WAS INDEED RESTRICTED TO JUST THIS JOURNAL.
WE CAN DO THAT EXACT SAME SEARCH IN E-DIRECT
WITH GENERALLY USING THE SAME SYNTAX.
SO I WOULD DO AN ESEARCH
I'M NOT GOING TO OVER THIS EVERY TIME, BUT WE'RE USING PUBMED. AND THEN I WANT TO BUILD THAT QUERY
AND I'M SEARCHING FOR MALARIA IN THE JOURNAL
JAMA.
THERE YOU SEE A SEARCH RESULTS.
A PMID COUNT I WOULD RETRIEVE FROM THAT.
ALSO JUST LIKE IN PUBMED WE CAN MANAGE THE SIZE
OF THE RESULT BY DATE RESTRICTION.
SAY I WANT TO MODIFY THE PREVIOUS SEARCH AND GET
ARTICLES PUBLISHED BETWEEN 2015 AND 2017.
I CAN DO THIS BY JUST WRITING THE DATE RESTRICTION INTO MY
QUERY ARGUMENT USING THE PUBLICATION DATE TAG
AS I WOULD IN PUBMED, WHICH I THINK MANY OF YOU ARE FAMILIAR WITH.
HOWEVER, ESEARCH OFFERS ARGUMENTS
THAT LET YOU LIMIT BY DATE RANGE.
THIS CAN BE USEFUL ESPECIALLY IF YOU HAVE A CAREFULLY CONSTRUCTED
AND VERY LONG SEARCH STRING, BUT WANT TO PUT SOME
ARTIFICIAL DATE RESTRICTIONS ON IT FOR TESTING,
BUT YOU DON'T ACTUALLY WANT TO START MUCKING AROUND IN THAT SEARCH QUERY THAT YOU BUILT.
AS I MENTIONED BEFORE, RATHER THAN RETYPING MY PREVIOUS COMMAND, I'M GOING TO USE THE UP ARROWS TO PULL
THE LATEST COMMAND I EXECUTED BACK INTO MY PROMPT.
LET ME CLEAR MY SCREEN FIRST.
SO I'M USING THAT UP ARROW, AND THERE YOU SEE MY PREVIOUS SEARCH.
AND THIS TIME I'M GOING TO USE THE DATE TYPE ARGUMENTS.
SO I'LL ADD THAT, WHICH IS SIMPLY DATETYPE
AND AS I SAID
WE WANT TO RESTRICT THIS BY PUBLICATION DATE, SPECIFICALLY
SO I NEED TO ADD MY PDAT DATETYPE
AND THEN THE RANGE I'M USING IS GOING TO BE BETWEEN
2015 AND 2017.
SO I ADD THAT BY ADDING A MINDATE AND A MAXDATE TO THIS
ARGUMENT, SO MY MINDATE IS 2012 -- PARDON ME, 2015. AND I WANT TO BE INCLUSIVE OF 2017
SO THAT'S MY MAX YEAR RANGE.
I RETRIVE 12. IF I DID THE SAME SEARCH IN PUBMED, WHICH I'VE DONE ALREADY,
AND I RESTRICTED THIS TO THE SAME DATE RANGE, WHICH IS 2015 TO 2017.
SEE I GET 12 RESULTS AS WELL
THAT'S USING THE SAME RESTROICTION ON THE DATE FIELD.
AND YOU CAN SEE IT HERE, TAGGED AS YOU WOULD IN PUBMED WITH THE PDAT DATES.
NOW THIS WORKS PERFECTLY WELL, BUT IF YOU'RE LIKE ME, THIS LONG STRING OF TEXT
IS GETTING HARD TO READ. AS YOU CAN SEE HERE, IT ALREADY COMES OFF MY SCREEN.
>> AS I SAID, A LOT OF THESE SCRIPTS ARE HUMAN
READABLE TO YOU CAN GO THROUGH AND
TROUBLESHOOT YOURSELF.
BUT IT GETS A LITTLE DIFFICULT TO READ.
IT MAKES IT EASIER FOR HUMAN READING IF YOU CAN BREAK IT UP INTO DIFFERENT
LINES AT LOGICAL BREAKPOINTS, TO SEGMENT THE DIFFERENT PARTS
AND HOW YOU'VE THOUGHT OF THEM AND BUILT THEM. WE CAN DO THIS BY USING THE BACKSLASH.
A BACKSLASH TELLS UNIX THAT THE COMMAND IS NOT DONE.
WHEN I HIT ENTER, UNIX IS EXPECTING THE REST
OF THE COMMAND LINE TO COME.
I JUST TYPE IN THE REST OF THE COMMAND AND HIT ENTER.
THIS CAN BE ESPECIALLY USEFUL WHEN YOU ARE COPYING
AND PASTING BLOCKS OF CODE.
YOU'RE GOING TO SEE IT IN A LOT OF MY EXAMPLES.
WHEN YOU SEE A LINE OF CODE END WITH THE BACKSLASH AND
SEE THE COMMAND CONTINUE TO THE NEXT LINE.
YOU CAN ALWAYS PUT THIS WHOLE COMMAND ON THE
SAME LINE IF YOU WANT TO MAKE SURE YOU REMOVE THE
ADDITIONAL BACKSLASHES BEFORE ENTERING.
I'LL JUMP BACK INTO MY SLIDES AGAIN.
SO WE'RE ABOUT TO DO OUR FIRST EXERCISE.
I SAID WE'D HAVE ABOUT FOUR EXERCISES IN THIS
CLASS AND BEFORE WE DO THERE'S ONE OTHER ITEM
THAT I'D LIKE TO DISCUSS AND THAT'S THE USE OF
QUOTES WITHIN A SEARCH STRING.
IF YOU HAVE TAKEN ANY OF OUR PUBMED CLASSES YOU
KNOW WE AVOID USING QUOTATION MARKS IN THE
SEARCH SINCE THEY GENERALLY TURN OFF
AUTOMATIC TERM MAPPING AND FUNCTIONS
DIFFERENTLY THAN WHAT YOU'RE USED TO IN A
STANDARD SEARCH ENGINE.
HOWEVER, THERE ARE TIMES WHEN YOU NEED TO DO A
LITERAL SEARCH AND WANT TO ENCLOSE IT IN QUOTES.
SAY WE WANT TO SEARCH FOR ARTICLES ABOUT CANCER
AND THE JOURNAL SCIENCE.
I'M GOING TO DO THIS IN PUBMED BECAUSE IT
DEMONSTRATES THE CONCEPT I'M TRYING TO SHOW YOU.
WELL -- SO AGAIN I'M GOING TO GET INTO A
BROWSER WINDOW IN PUBMED.
AND AGAIN, I WANT TO LOOK FOR ALL OF THE ARTICLES
DEALING WITH CANCER.
AGAIN I HAVE A SUBSCRIPTION TO THE
JOURNAL "SCIENCE."
THE SAME SEARCH YOU SAW ME DO BEFORE.
SPECIFIED WHAT TERMS I'M LOOKING FOR, CONCEPTS
IN PUBMED, AND I'VE JOINED THAT TO SPECIFIY WHAT
JOURNAL I'VE USED.
>> SORRY.
AND ATM HAS DONE ITS THING.
WE'RE MAPPING TO NEOPLASMS WHICH IS CORRECT
AND WE'RE STILL SEARCHING CANCER IN ALL FIELDS.
AND THEN YOU'LL SEE SCIENCE IS HERE AS A JOURNAL.
THEN I SEE A COUPLE OF OTHER JOURNALS HERE.
THAT'S BECAUSE JOURNALS ARE LISTED AS AN
ALTERNATIVE TITLE AND THAT'S AFFECTING
THE RETRIEVAL.
SO TO AVOID THIS I CAN COME UP AND I CAN
ENCLOSE SCIENCE IN QUOTES TELLING PUBMED,
I TAG THE JOURNAL AND I WANT TO LOOK FOR JUST
THIS IN PUBMED.
NOW YOU SEE I'M ONLY SEARCHING
IN THE JOURNAL SCIENCE.
THIS CAN BE TRICKY IN E-DIRECT.
WE'RE ALREADY CLOSING OUR QUERY.
YOU WANT TO CONSTRUCT THE SAME SEARCH IN E-DIRECT
TO SPECIFY JUST THAT JOURNAL
SINCE UNIX WILL GET CONFUSED WE WANT TO
USE BACKSLASHES BEFORE
EACH QUOTE THAT WILL SIGNIFY IT IS NOT
CLOSING THE QUERY STRING.
THE EASIEST THING TO REMEMBER ABOUT THIS IS
ANYTIME YOU USE QUOTES WITHIN THAT SEARCH QUERY, PUT THE BACKSLASH RIGHT BEFORE THE QUOTE
AT THE BEGINNING, AND THEN WHEN YOU CLOSE THEM, PUT IT BEFORE AT THE END.
IF THIS CONFUSES ANYONE I APOLOGIZE FOR THAT.
OKAY.
SO I SAID WE'RE GOING TO DO OUR FIRST EXERCISE,
I'D LIKE TO KNOW HOW MANY SPANISH LANGUAGE ARTICLES ABOUT
DIABETES ARE IN PUBMED AND I'D LIKE YOU TO DO THIS
USING EDIRECT NOT THE PUBMED NOT THE INTERFACE.
JUST AS A HINT, THERE'S THE LANGUAGE TAG ON THE SCREEN, THAT YOU'LL PROBABLY NEED TO USE.
WHEN YOU'RE DONE GO AHEAD AND PUT A GREEN CHECK IN THE
CHAT AND WE'LL MOVE ON.
I'M GOING TO GIVE YOU A COUPLE OF MINUTES.
I'M GOING TO MUTE AND I'LL CHECK BACK IN WHEN
YOU HAVE COMPLETED THIS.
OK, IT LOOKS LIKE ABOUT HALF OF YOU HAVE FINISHED THIS
OR AT LEAST PUT IN THE CHAT THAT YOU DID.
I KNOW SOME PEOPLE DON'T HAVE E-DIRECT AND THAT'S ONE OF THE
REASONS WE'RE GOING TO GO OVER IT AT THE END OF CLASS.
THAT'S ALSO WHY I'M GOING TO DEMONSTRATE THESE.
SO, LET ME GET OUT OF PUBMED HERE.
AND I'M CLEARING MY E-DIRECT SCREEN.
THE QUESTION WE WERE POSED IS HOW MANY
SPANISH ARTICLES ABOUT DIABETES ARE IN PUBMED.
SO WE'LL START WITH THE SEARCH.
WE'LL DO A SEARCH IN PUBMED.
WE'RE LOOKING FOR CITATIONS ABOUT DIABETES.
ONLY IN SPANISH.
I'M TAGGING IT HERE WITH THAT LANGUAGE TAG.
AND I NEED TO CLOSE MY QUOTES.
YOU CAN SEE WE'VE RETRIEVED 8,645 AND I TOLD YOU BEFORE
THAT WE CAN ALWAYS VERIFY OUR WORK
ESPECIALLY WHEN YOU'RE FIRST GETTING STARTED.
I'M GOING TO DO THIS BY ENTERING MY SEARCH PHRASE: DIABETES.
I SAID WE'RE GOING TO FILTER BY SPANISH.
SO I'M USING THE SIDE FILTERS TO ACCOMPLISH
THE SAME THING.
AND NOW I JUST ENACTED IT.
AND 8,645.
THAT WAS A PRETTY EASY ONE.
ANOTHER EXERCISE TO REINFORCE SOME THE E-SEARCH YOU ARE DOING.
I'D LIKE TO KNOW HOW MANY ARTICLES WERE WRITTEN
BY BH SMITH BETWEEN 2012 AND 2017 INCLUSIVELY?
JUST A COUPLE OF HINTS THAT I DIDN'T PUT ON THE SLIDE.
YOU CAN USE THE AUTHOR TAG.
AND FOR THOSE YEARS BETWEEN 2012 AND 2017, I THINK YOU
KNOW THERE ARE TWO DIFFERENT WAYS THAT YOU
CAN ACCOMPLISH THAT.
THIS ONE IS A LITTLE BIT MORE COMPLICATED.
I'LL GIVE YOU AN EXTRA MINUTE OR TWO ON THIS.
I'M GOING TO MUTE.
IF THERE ARE ANY QUESTIONS, PLEASE PUT THEM
IN THE CHAT PANEL AND WE'LL GET TO YOU.
OK TERRIFIC! IT LOOKS LIKE A LOT OF PEOPLE ARE FOLLOWING ALONG. LET'S GO AHEAD AND DO THIS ONE TOGETHER.
I'M GOING TO CLEAR MY SCREEN AGAIN. WE WANTED TO DO A SEARCH FOR ALL THE CITATIONS FROM BH SMITH.
WE'RE SEARCHING PUBMED AND OUR SEARCH IS GOING
TO BE FOR A SPECIFIC AUTHOR AND THE AUTHOR'S
NAME AS I SAID, I COULD HAVE GIVEN YOU THAT AS A HINT,
TO PROPERLY CONSTRUCT THIS SEARCH FOR THIS AUTHOR
WOULD BE SMITH LAST NAME, INITIALS ONE OR TWO.
IF HAVE YOU TWO, USE THEM.
I'M TAGGING THEM WITH AUTHOR AND THEN I AM GOING TO USE
THAT DATE TYPE ARGUMENT, BECAUSE I DON'T WANT TO
TOUCH THAT SEARCH THAT I DID.
I'M USING PDAT BECAUSE I
ONLY WANT TO RETRIEVE THE CITATIONS THAT HE'S
PUBLISHED BETWEEN 2012 AND 2017 INCLUSIVELY.
SO MY MINDATE IS 2012, AND I WANT TO BRING THAT ALL THE WAY UP TO 2017.
AND AGAIN MY DATE TYPE.
YOU CAN SEE I DID SOME DEBUGGING
OF MY OWN SCRIPT.
WE TALKED ABOUT HOW SENSITIVE EVERYTHING IS
FOR EXAMPLE, IF I HADN'T PUT THAT SPACE IN, WE WOULD NOT
HAVE RETRIEVED ANYTHING.
AND THERE YOU SEE OUR COUNT: 149.
WE COULD HAVE DONE THAT SAME SEARCH PUTTING THE
PDAT AS PART OF OUR SEARCH QUERY STRING.
IF YOU'RE STILL WORKING ON THE EXERCISE I'LL ASK
YOU TO SET IT ASIDE FOR NOW BECAUSE WE'RE GOING
TO MOVE ON TO THE NEXT TOPIC.
ANSWERS TO THESE ARE AT THE BOTTOM OF THE COURSE
MATERIAL TEXT FILE YOU HAVE AND WE ENCOURAGE
YOU AFTER THE CLASS TO GO BACK AND LOOK AT THEM
AND REVIEW THEM AND ANSWER THEM YOURSELVES.
IF YOU STILL HAVE PROBLEMS YOU CAN CONTACT
US THROUGH E-MAIL OR WAIT UNTIL THURSDAY AND
SARAH'S GOING TO HAVE A COUPLE MOMENTS AT THE
BEGINNING OF THE CLASS TO GO OVER ANY QUESTIONS
FROM THIS ONE THAT YOU MAY HAVE.
SO WHEN WE STARTED TALKING ABOUT COMMANDS I
STARTED WITH E-SEARCH AND NOW OUR NEXT COMMAND
IS E-FETCH AND I'M GOING TO HAVE THE SAME
SOFTBALL QUESTION OUT THERE TO GET EVERYBODY
WARMED UP.
WHAT DO YOU THINK E-FETCH DOES?
WE'RE ASKING THIS ONE -- WE'RE SPECIFYING THAT WE'RE
RETRIEVING FULL RECORDS FROM PMIDs AND WE RETRIEVED THOSE
IN A VARIETY OF FORMATS AND THEY'RE IN THE
DOCUMENTATION.
I'LL REVIEW THE MOST COMMON ONES IN THIS
COURSE BUT THERE ARE DIFFERENT FORMATS AND
EACH DATABASE RETRIEVES SLIGHTLY DIFFERENT
FORMATS DEPENDING ON WHAT MAKES SENSE FOR THAT
DATABASE THAT YOU'RE QUERYING.
SO I'M GOING TO DEMO E-FETCH.
AND I'M GOING TO RETRIEVE THE ABSTRACT OF THE PMID
USE SO IF I START WITH MY COMMAND AGAIN AND I
SAID THAT'S E-FETCH AND WE WANT TO USE THE SAME
DATABASE PUBMED.
THERE'S NO CHANGE THERE.
NEXT I WANT TO ADD THE ARGUMENT I.D.
AND THIS SPECIFIES WHAT RECORDS YOU WANT
TO RETRIEVE.
I SAID I WANT A SPECIFIC PMID I KNOW E-FETCH HAS
THEM AS A UNIQUE IDENTIFIER.
I WANT TO RETRIEVE PMID 25359968 --
AND THEN I SAID THE FORMAT THAT I'M MOST FAMILIAR
WITH FROM USING PUBMED IS ABSTRACT.
SO I SPECIFY THAT IN MY FORMAT ARGUMENT.
THEN I RETRIEVE THIS RECORD.
WHAT YOU'RE SEEING IS AN ABSTRACT VIEW WITHIN
PUBMED AND THE JOURNAL NAME, PUBLICATION DATE, AUTHOR'S NAME,
THEIR AFFILIATION INFORMATION,
AND YOU SEE THE
ABSTRACT AND THEN AT THE BOTTOM SOME OTHER
INFORMATION YOU SEE IN THE PUBMED DISPLAY SUCH
AS THE PMCID AND DOI.
USING MY UP AND DOWN ARROWS, I SAID THAT I CAN GET THIS IN A COUPLE OF DIFFERENT FORMATS.
>> MANY ARE FAMILIAR WITH THE TAGGED FORMAT
OF OUR RECORDS.
I CHANGE THE FORMAT TO MED LINE INSTEAD OF
ABSTRACT, I HIT ENTER AND NOW YOU'LL SEE THE
TAGGED VIEW OF THAT RECORD.
HERE ARE ALL THE DIFFERENT TAGS WITHIN PUBMED AND THEIR
ASSOCIATED METADATA.
I CAN ALSO CHANGE THIS FORMAT TO XML IF I WANT TO.
SARAH IS GOING TO TALK MORE ABOUT THE
STRUCTURE OF OUR DATA IN XML IN THURSDAY'S CLASS, BUT WHAT YOU'RE
SEEING WHEN YOU RETRIEVE THE XML FORMAT IS THE
FULL XML OF THIS RECORD IN PUBMED.
SO YOU'RE SEEING MORE DATA THAN YOU GOT IN THE
OTHER VIEWS BUT AT THE HEART OF IT, WHAT YOU'RE
LOOKING FOR IS STILL HERE, YOU'RE SEEING THE
PMID, THE JOURNAL THAT IT'S FROM, ABSTRACT IN TEXT.
YOU'LL SEE YOUR AUTHORS LISTED AND MESH TERMS.
IF IT'S A MEDLINE RECORD.
WE'LL GO OVER THE XML OF THE RECORDS IN MORE DETAIL ON THURSDAY AND THROUGHOUT
COURSE AS WE'LL WORK IN THE DATA QUITE A BIT.
THE LAST FORMAT I WANTED TO SHOW YOU IS THAT I
CAN RETRIEVE THIS UNIQUE IDENTIFIER AS A UNIQUE
IDENTIFIER.
WE'LL TALK IN A MINUTE ABOUT WHEN YOU'D WANT
TO DO THIS.
IF YOU WANT TO RETRIEVE THE RECORDS FOR MULTIPLE
CITATIONS -- I'LL CLEAR MY WINDOW HERE --
YOU CAN LIST YOUR PMIDS WITH A COMMA SEPARATING THEM. AS YOU CAN SEE,
THERE'S NO SPACES BETWEEN.
AND I'VE RETRIEVED THREE DIFFERENT PMIDS.
YOU CAN SEE THEM LISTED HERE:
ONE, TWO AND THREE.
A NOTE OF CAUTION.
THIS RETRIEVAL AND DISPLAY NEEDS TO BE DONE CAUTIOUSLY AND I'LL
SHOW YOU AN EXAMPLE OF WHEN THIS CAN BE A LITTLE BIT OFF-PUTTING.
MY COLLEAGUE MIKE FOUND THIS RECORD WHICH MAKES THE
THE POINT VERY WELL.
26024162 AND I'M GOING TO RETRIEVE THAT
IN THE ABSTRACT VIEW.
SO THE FIRST CLUE WAS,
YOU SAW WHEN I HIT RETURN THE RETRIEVAL TOOK A MOMENT.
WHICH YOU DIDN'T SEE WITH MANY OF THE OTHER RECORDS.
AND AS I SCROLL THROUGH, YOU'LL SEE A HUGE LIST
OF AUTHOR NAMES.
WHICH MAKES THIS RATHER DIFFICULT TO USE THIS RECORD.
AND I DON'T KNOW WHAT I WOULD DO WITH IT ONCE I
GOT IT OUT OF HERE IN THIS FORMAT.
JUST A CAUTIONARY TALE, BE CAREFUL WHEN YOU'RE TESTING.
THIS MIGHT BE A REASON WHY YOU WOULD JUST WANT TO
TO RETRIEVE A COUNT OR A LIST OF UIDs INSTEAD OF
ACTUALLY RETRIEVING THE FULL RECORD.
SO, I THINK THAT WAS A PRETTY STRAIGHTFORWARD
COMMAND THAT WILL TAKE US TO OUR NEXT EXERCISE.
I'D LIKE YOU TO TELL ME ONLY USING E-DIRECT WHO
IS THE FIRST AUTHOR LISTED IN THE PUBMED
RECORD 26287646.
I DON'T REALLY HAVE HINTS FOR YOU ON THIS ONE.
IT'S A STRAIGHTFORWARD RETRIEVAL.
I'LL GIVE YOU A COUPLE MINUTES AND THEN I'LL
CHECK BACK WITH YOU AND WE'LL GO OVER
IT TOGETHER.
AND IF YOU HAVE ANY QUESTIONS PUT THEM IN THE
CHAT AS WE'RE WORKING ON THIS ONE.
LET'S GO AHEAD AND WORK ON THAT ONE.
SO HERE'S MY EDIRECT THAT YOU'VE BEEN LOOKING AT ALL HOUR.
SO WE'RE LOOKING FOR THE FIRST AUTHOR LISTED ON
THE PUBMED RECORD, 26287646.
WE'RE GOING TO FETCH THAT RECORD FROM PUBMED.
WE'VE BEEN GIVEN THE PMID.
I ALSO TAKE A MOMENT TO THINK ABOUT WHAT FORMAT I'D WANT TO SEE THIS IN.
I'M ALWAYS WORKING IN PUBMED IN THE ABSTRACT VIEW, SO THAT'S WHAT I WANT TO RETRIEVE IT IN.
YOU'LL SEE I GOT THE RECORD LOOKING
AT THE PMID.
THEN WE HAVE P BRENNAN, THE DIRECTOR OF NATIONAL
LIBRARY OF MEDICINE WHERE I WORK.
I DON'T THINK I'LL HAVE QUESTIONS ON THAT ONE.
IF YOU'RE STILL WORKING ON IT THOUGH, I'D ASK
YOU TO PUT IT ASIDE AND GO BACK TO THE SLIDE SO
BEAR WITH ME FOR A MOMENT.
I SAID WHAT MAKES UNIX AND EDIRECT POWERFUL AND EFFICIENT TOOLS, IS THE ABILITY
TO COMBINE COMMANDS TOGETHER, USING THE RESULTS OF ONE AS THE INPUT OF ANOTHER.
WE'RE TALKING ABOUT CREATING PIPELINE OF DATA
THROUGH A SERIES OF COMMANDS AND WE DO THAT
BY USING THE PIPE CHARACTER I SHOWED YOU
AT THE BEGINNING.
RIGHT NOW WE'VE ONLY TALKED ABOUT TWO
COMMANDS THAT FIT TOGETHER LIKE THIS,
E-SEARCH AND E-FETCH.
WE'LL GO MORE THROUGH THE COURSE.
IF YOU PIPE THE SEARCH RESULTS FROM AN E-SEARCH INTO AN E-FETCH
WE CAN GET ALL OF THE RESULTS FROM OUR SEARCH AS A LIST OF PMIDs, IN TEXT ABSTRACT, OR IN XML,
OR WHATEVER FORMAT WE WANT AND IS SUPPORTED IN
THIS DATABASE.
USING PIPE IS RATHER EASY.
TYPE THE FIRST COMMAND AND SPACE AND TYPE PIPE.
THE PIPE CHARACTER.
THEN TYPE THE SECOND COMMAND.
SO FOR EXAMPLE I'M SHOWING ON THE SCREEN
YOU'LL SEE MY FIRST COMMAND IS I'M SEARCHING
AND I'M SEARCHING FOR ASTHENOPIA AS A MESH HEADING,
SUBHEADING OF NURSING. ANYTHING I RETRIEVE FROM THAT, I WANT TO FETCH, AND I JUST WANT TO RETRIEVE THE UIDS.
SO I'VE PIPED THAT OVER TO EFETCH, AND THIS WILL ALSO PIPE OVER THE DATABASE, SO IT LETS EFETCH KNOW
WE'RE IN PUBMED AND WE'RE GOING TO
BE RETRIEVING THE PMIDs AS OUR IDs IN THAT DATABASE.
I SAID BEFORE I'D SHOW YOU OR TELL YOU WHEN
THIS IS A VERY CASE FOR JUST RETRIEVING IN A
FORMAT OF UID.
BECAUSE YOU'RE NOT SURE WHAT YOU'RE RETRIEVAL'S
GOING TO BE AND THE NUMBER OF RECORDS
YOU'LL GET.
AND JUST TO KEEP THIS READABLE WE COMBINED THE
BACKSLASH WITH THE PIPING COMMAND JUST TO
KEEP IT MORE READABLE.
YOU CAN SEE WHAT'S GOING ON HERE.
YOU CAN SEE MY DIFFERENT SEARCH AND PIPED OVER
TO E-FETCH.
WE'LL DEMO THAT AS WE GO THROUGH THE CLASS.
WE'RE ABOUT TO DO OUR FINAL EXERCISE.
BUT BEFORE I DO THAT I'D LIKE TO GO OVER TIPS FOR SCRIPTING.
THE NEXT EXERCISE WILL BE THE FIRST MULTI-COMMAND
SCRIPT BUT FIRST I WANT TO GIVE YOU TIPS TO
THINK ABOUT IN THE EXERCISE.
THINGS CAN GET UNWIELDY QUICKLY.
YOU WANT TO KEEP YOUR INITIAL QUERY MANAGEABLE
AND CHECK THAT THE SEARCH COMMAND IS NOT TOO BIG.
YOU WANT TO COMBINE COMMANDS SLOWLY, MAKING SURE YOUR TEST SET REMAINS MANAGEABLE
AND LASTLY, WORK IN SMALL STEPS, BUILDING UP SLOWLY.
KEEP ALL OF THESE TIPS IN MIND AS YOU WORK ON THE HOMEWORK,
AS WE WORK THROUGH THIS COURSE, TOO.
THESE ARE SOME GREAT TIPS TO KEEP IN MIND.
SO WITH ALL THAT IN MIND, I'D LIKE TO KNOW HOW DO
WE GET A LIST OF PMIDs.
I WANT THE LIST NOT THE RECORDS FOR ALL THE
ARTICLES WRITTEN BY BH SMITH BETWEEN 2012
AND 2017.
REMEMBER YOU'RE UP AND DOWN ARROWS.
TO ACCESS THE PREVIOUS COMMANDS, BECAUSE THAT SEARCH COULD LOOK PRETTY FAMILIAR,
AND YOU KNOW YOU PROBABLY ALREADY HAVE A GOOD SCRIPT WRITTEN FOR THAT.
ANOTHER THING IS I ONLY WANT THE PMIDS.
SO THAT UID FORMAT IS VERY HELPFUL.
I'M GOING TO GIVE YOU FIVE MINUTES
TO WORK ON THIS ONE
BECAUSE IT'S A LITTLE MORE COMPLICATED THAN
SOME OF OUR OTHER EXERCISES.
WE'LL COME BACK AND WORK ON IT TOGETHER.
AS YOU FINISH, YOU CAN GO AHEAD AND JUST PUT A
GREEN CHECK IN THE PARTICIPANTS WINDOW.
>> OKAY.
IT LOOKS LIKE A LOT OF PEOPLE GOT SOME RESULTS.
AND HOPEFULLY THAT'S GOOD.
LET'S GO OVER THIS TOGETHER.
SO HERE I HAVE MY E-DIRECT WINDOW OPEN AND
WE WANTED TO KNOW -- WE WANTED TO GET ALL THE
UIDs FOR CITATIONS FROM THE AUTHOR, BH SMITH,
BETWEEN 2012 AND 2017.
WHEN YOU WORK IN PUBMED FOR AUTHOR SEARCHES THE
INDEX IS BUILT BY THE LAST NAME AND THEN
INITIALS OF THE AUTHOR.
SO YOU'LL HAVE MULTIPLE AUTHORS WITH THE NAME
SMITH AND THE INITIALS BH.
THAT'S WHY I SAID IF YOU WANT TO BE AS SPECIFIC AS
POSSIBLE YOU ENTER BOTH INITIALS BUT THE SEARCH CAN
BE CONDUCTED THE SEARCH WITH THE LAST NAME
AND FIRST INITIAL.
SO, IT IS EXPECTED THAT YOU WILL SEE MANY AUTHORS WITH THE NAME
OF SMITH WITH THE INITIALS BH WHO ARE NOT THE SAME PERSON.
DISAMBIGUATION IN PUBMED IS SOMETHING THAT
WE'RE ALWAYS WORKING ON.
AND WHY YOU NEED TO LEARN TO SEARCH AND
PARSE THE AFFILIATION DATA THAT'S PROVIDED IN PUBMED
THAT'S OFTEN USED AND VERY HELPFUL IN
DISAMBIGUATION OF AUTHORS.
IT'S SOMETHING WE'LL GET INTO AS YOU GO THROUGH THIS COURSE.
WHICH IS VERY HELPFUL.
SO BACK TO THE TASK AT HAND.
LET'S GET THE UIDs FROM THE AUTHOR OF BH SMITH
BETWEEN 2012 AND 2015.
SO THE FIRST THING WE NEED TO DO IS WE NEED TO
SEARCH PUBMED.
SO WE'LL BUILD THAT SEARCH.
WE WANTED TO -- WE'RE SPECIFICALLY SEARCHING
FOR ARTICLES FROM THE AUTHOR SMITH WITH THE
INITIALS BH AND WE KNOW WE ONLY WANT THE AUTHOR.
I AM GOING TO RESTRICT THIS TO A CERTAIN DATE FIELD SO
I JUST GET THE MOST CURRENT LITERATURE FROM THIS AUTHOR.
I ALSO KNOW I DON'T WANT TO MESS AROUND WITH THE QUERY STRING
TOO MUCH BECAUSE I HAVE A GOOD AUTHOR LIST HERE.
SO I'M GOING TO AMEND THAT.
BUT BEFORE I DO, I'M GOING TO USE THE PDAT --
DATE TYPE ARGUMENT.
AND THAT CAN GET A LITTLE LONG AND HARD TO
READ SO I'LL BREAK IT UP AND KEEP FOLLOWING.
I PUT THE SPACE AND BACKSLASH AND THEN ENTER.
YOU CAN SEE I GET BACK TO THE COMMAND PROMPT.
NOTHING'S BEEN EXECUTED.
E-DIRECT IS WAITING FOR ME IT PUT IN THE REST OF
THIS COMMAND.
AND I SAID THAT I WANTED TO RESTRICT IT BY DATE TYPE.
SPECIFICALLY THE PUBLICATION DATE.
I NEED THAT BETWEEN A RANGE OF 2012 AND I
WANTED TO INCLUDE 2017 THEN I WANT TO RETRIEVE
THE PMIDs.
I'M GOING TO PIPE THAT OVER AND FETCH JUST THE UIDs.
AGAIN, WE HAVE
TWO COMMANDS THAT WE'RE EXECUTING HERE, AND WE'RE PIPING THROUGH THE DATABASE OVER AS WELL TO EFETCH
AND THE RETRIEVED UIDS, AND WE'RE SPECIFYING THAT FOR A DATE TYPE.
I'VE BROKEN THIS UP SO I CAN READ IT, BY USING THAT BACKSLASH AND THE SPACE
AND HERE IS OUR LIST OF UIDs, AND HERE'S A GREAT EXAMPLE OF WH
THIS IS THE BEST FORMAT TO RETRIEVE THE DATA IN.
I HAVE NO IDEA HOW LONG THE RECORDS ARE OR HOW MUCH
SPACE THEY'D TAKE UP. I CAN NOW TAKE THIS AND GO INVESTIGATE WHAT I WANT TO DO WITH THIS DATA.
DO I HAVE QUESTIONS ON THAT EXERCISE SPECIFICALLY?
OKAY.
WE'RE RUNNING OUT OF TIME AND I'D LIKE TO LEAVE A
LITTLE BIT OF TIME FOR QUESTIONS.
LET ME GO BACK TO MY SLIDES.
I'D LIKE TO PUT A PLUG IN HERE FOR THE NEXT CLASS
FOR SARAH AND SHE'LL TALK ABOUT HOW TO GET
DATA ELEMENTS OUT OF XML AND TABULATE THEM USING THE
XTRACT COMMAND.
SHE'S GOING TO EXPLAINA LITTLE BIT MORE OF THE RAW XML OF OUR DATA
AND THAT'S THURSDAY AT THE SAME TIME.
IF BETWEEN NOW AND THURSDAY YOU'RE CURIOUS
ABOUT UNIX, OR YOU'D LIKE TO KNOW A LITTLE BIT ABOUT IT, HERE'S A URL FOR AN NCBI NOW VIDEO.
IT'S ABOUT 30 MINUTES IT'S GIVEN BY ONE OF OUR COLLEAGUES.
IT'S A GREAT LITTLE OVERVIEW.
IT'S REALLY APPROACHABLE AND I REALLY RECOMMEND LOOKING
AT THAT IF YOU HAVEN'T ALREADY.
IF YOU WANT MORE INFORMATION ABOUT
E-DIRECT, CHECK OUT OUR DOCUMENTATION AT THE
WEBSITE YOU SEE ON THE SLIDE.
IT'S THE SAME ONE YOU WENT TO SIGN UP FOR AND
THE SAME ONE YOU GO TO GET YOUR HOMEWORK AND
SLIDES AND RECORDINGS AND EVERYTHING ARE THERE.
IF YOU HAVE QUESTIONS BETWEEN NOW AND THURSDAY
YOU CAN HIT THE CONTACT BUTTON AT THE TOP OF
YOUR SCREEN AND E-MAIL US ON THE FIRST SLIDE ON
NLM TRAINERS AT NIH.gov.
WE'LL TRY TO GET BACK TO YOU ASAP BUT IF WE DON'T
SARAH WILL PUT TIME IN THE BEGINNING OF HER
CLASS TO GO OVER QUESTIONS AND ANSWER
SOME ON THE HOMEWORK.
WHILE YOU'RE ON THE CONTACTS PAGE IF YOU GO
THERE, PLEASE CONSIDER SIGNING UP FOR THE
UTILITIES MAIL LIST.
WE PUT OUT ANNOUNCEMENTS OCCASIONALLY FOR UPDATES
AND NEW CLASSES WE'RE ADDING TO THE COURSE AS WE GO.
I'D LIKE TO STOP THE RECORDING AND GO OVER
THE HOMEWORK.
ACTUALLY, LET ME KEEP GOING.
WE HAD HOMEWORK FOR EACH ONE OF THESE CLASSES ON
THE HONOR SYSTEM BUT WE THINK THESE QUESTIONS AT
THE END WILL HELP REINFORCE THE TRAINING
FOR TODAY AS WELL AS GET YOU READY FOR
THE NEXT CLASS.
SO I SAID THAT WE HAVE ALL OF OUR RESOURCES LISTED
ON THE INSIDERS GUIDE TO ACCESSING NLM DATA
AND I HOPE YOU CAN SEE THIS ON YOUR SCREEN.
I'M GOING TO GO DOWN HERE TO ONLINE CLASSES.
HERE'S THE WELCOME TO E-UTILITIES FOR PUB MED
CLASS AND THERE'S A RECORDING IN HERE.
IF YOU CLICK. I'M GOING TO GO TO EDIRECT FOR PUBMED, WHERE WE'RE
IN CURRENTLY.
HERE YOU SEE THE CLASSES.
AND THE RECORDINGS FROM THE PREVIOUS ONES.
GO UNDER CLASS MATERIAL, COME DOWN TO HANDOUT AND
WE'RE IN PART ONE.
SCROLL DOWN AND I USUALLY GO PAST THIS SO
PARDON ME IF I DO.
WE'RE GOING TO SEE HOMEWORK FOR PART ONE.
YOU'LL SEE THERE'S FIVE QUESTIONS.
HOPEFULLY AT THE TOP HERE WE SAY WHERE THE ANSWERS
ARE FOR THESE AND OF COURSE IF YOU HAVE ANY
QUESTIONS AS WE GO WE'LL BE MORE THAN HAPPY TO
ADDRESS THEM NEXT CLASS AND IT LOOKS LIKE I HAVE
A FEW MINUTES FOR QUESTIONS.