Dr Fernando Loizides, Faculty of Science and Engineering

Dr Fernando Loizides is a Lecturer (Assistant Professor) in Computer Science at the School of Mathematics and Computer Science at the University of Wolverhampton, UK. He is part of a research group in emerging interactive technologies (EITLAB.NET).

His main research lies in Information Interaction, HCI and Digital Libraries, focusing on Information Seeking, Information Architecture and User Experience, with a special interest on user interfaces.

He has published work in international journals and conferences, taken part in EU funded as well as national funded projects, edited books and proceedings and successfully secured personal EU and national funding.

Fernando also has applied development experience, with one of his mobile applications winning the digital championship social impact award and several other applications being commercialised.

Creating International Standards and Benchmarks for Patent Examination Systems Through Eye-tracking Testing

The protection of intellectual property through patents is at the forefront of today’s technological advancements. Disputes over patent infringement value in the billions yearly.  A single patent examiner, assigned to review a patent application and ultimately make a decision on whether to accept or reject it. Patent examiners search for patents which are similar to the submitted proposal to determine whether the claim meets the patentability requirements in terms of novelty and is therefore eligible to be approved for a patent. Patent examiners each have their own methodologies to examine the patent applications which they adopt through their experience over their time at the  patent office. No standardized examining methodologies exist and minimal training and support is provided to the examiners. 

We aim to use state of the art technology such as eye-tracking to achieve the following goals:  

1) Setting an international standard for patent examination methodologies for training and assessing examiners.

2) Creating International standards by which patent examination systems are built, tested and benchmarked

From the requirements, a prototype system for patent triage will be developed and tested.

ERAS Executive Summary 2016/17

 1) Introduction

The protection of intellectual property through patents is at the forefront of today’s technological advancements. Disputes over patent infringement value in the billions yearly. The examination of intellectual property applications takes place in patent offices worldwide. A single patent examiner, assigned to review a patent application and ultimately make a decision on whether to accept or reject it. Patent examiners search for patents which are similar to the submitted proposal to determine whether the claim meets the patentability requirements in terms of novelty and is therefore eligible to be approved for a patent. Patent examiners each have their own methodologies to examine the patent applications which they adopt through their experience over their time at the patent office. No standardized examining methodologies exist and minimal training and support is provided to the examiners.

2) Background

The exponential increase in patents and patent applications make it necessary for the patent examiner to attempt faster passes over patent documents to find related ones to compare to the application. Faster patent examination is also indirectly encouraged by the need to reach a predetermined ‘quota’ of examined patents per year, with further benefits if employees exceed that number. The examiners therefore suffer from information overload; namely, having too much information to have to go through. The process that one undertakes to traverse through all the information to gain meaning and also decide the relevance of each information element to one’s information need is dubbed ‘information triage’. More specifically, ‘document triage’ is the process that one undertakes to assess the relevance of individual documents in a corpus to an information need. The combination of fast triage and visual human scrutiny has been shown to lead to less than optimal decisions on document relevance. Recently, the European Union has funded the development of assistive software for patent examiners in the European Patent Office (EPO). The requirements elicitation and the evaluation of this system as well as other such systems worldwide are not currently in existence giving us a large opportunity for a systematic analysis and creation of guidelines for such business critical systems.

3) Aims & objectives

1)      Setting an international standard for patent examination methodologies for training and assessing examiners through the introduction of a better understanding and modelling of the human computer interaction process during patent examination.

2)      Creating International standards by which patent examination systems are built, tested and benchmarked by analysing the interaction and requirements engineering from the users. The efficacy of using state of the art interactive technology such as eye tracking will also be investigated.

4) Methods

We used a mixed methods approach. We use the quantitative methods to create our benchmarks and the qualitative methods to compliment the quantitative information and help us understand user requirements.

The quantitative studies included eye-tracking with spatial recognition and time, measuring the time gaze factor and therefore be able to identify perceived relevance and importance. We are also able to directly identify system elements used and those that are underutilised by the examiners.

Qualitative studies help to identify reasons for specific behaviours as well as requirements assessment by interview. In this phase, all stakeholders are included to address issues such as business critical requirements as well as usability factors. Semi-structured interviews as well as focus groups were used.

5) Research design

3 testing sessions at the European Patent Office took place to gather data.

The first used an eye-tracking methodology in conjunction with semi-structured interviews from the examiners themselves. We were able for the first time to report on the exact patent examiner action and precise visual attention during their patent examination. From these observations we proposed detailed metrics that can provide benchmarks beyond the log metrics in existence currently.

The second test elicited requirements from a larger set of stakeholders such as the business operations and managerial staff. This took place in focus groups and with semi-structured interview techniques. Semi-structured interviews with the examiners were also taken to enrich the findings.

From the requirements, a prototype system for patent triage was developed and tested.

The third and final test at the European Patent Office, included giving the bespoke prototype tool to the examiners and stakeholders and to begin a user-centred design approach to improve it constantly for the open use of the system by the examiners as well as the public. Feedback in the form of semi0structured interview notes were taken.

Future work includes releasing the 2^nd version based on the feedback and iteration is being planned.

6) Summary findings

The project has been successfully completed. The outcomes and findings are:

Outcomes / Deliverables:

1)      Paper published in ElPub 2017 conference (peer reviewed and indexed) based on the algorithm and methodology devised for searching textual documents.

2)      Paper published in ElPub 2017 conference (peer reviewed and indexed) based on the stakeholder and examiner model behaviour and requirements for non-patent prior art searching.

3)      Accepted journal paper in Information Services in Use.

4)      Successful funding (80,000GBP) for the use of the patentable research into creating a patentable product based on VR research.

5)      Invited talk to present the research to the International Federation of Information Processing (TC-13 – Human Computer Interaction Group)

6)      Invitation for training the examiners and white paper on the metrics and models for prior art searching techniques.

7)      Journal article being prepared for JASIST in collaboration with the Patent Office.

8)      Working prototype for NPL searching

7) References

[1]    Fernando Loizides, Barrou Diallo, Andrew Pollard, Aekaterini Mavri, Increasing the Discovery and Use of Non-Patent Literature (NPL): Scientific Publications in Patent Examination,  pp 211 – 216 DOi 10.3233/978-1-61499-769-6-211

[2]    Fernando Loizides, Sam A. M. Jones:
Insights from Over a Decade of Electronic Publishing Research. ELPUB 2016: 119-124

[3]    Fernando Loizides, Sam A. M. Jones:
A methodology for digitally exploring electronic publication content. Inf. Services and Use36(3-4): 211-214 (2016)

[4]    Bawden, D., & Robinson, L. (2009). The dark side of information: overload, anxiety and other paradoxes and pathologies. Journal of information science, 35(2), 180-191.

[5]    Buchanan, G and Loizides. F 2007. Investigating document triage on paper and electronic media. In Proceedings of the 11th European conference on Research and Advanced Technology for Digital Libraries (ECDL'07), Springer-Verlag, Berlin, Heidelberg, 416-427.

[6]    Loizides, Fernando. "Information seekers’ visual focus during time constraint document triage." Theory and Practice of Digital Libraries. Springer Berlin Heidelberg, 2012. 25-31.