Add support for context analysis - Status of a named entity
Piyush13y opened this issue · comments
Is your feature request related to a problem? Please describe.
As part of our medical NLP pipeline, we have implemented one part of the context analysis module i.e. NegationContextAnalyzer which detects negation of named entities. Now, we need the other aspect of it, StatusContextAnalyzer which should work analogous to its CTakes counterpart. This processor should get the status of every identified named entity in the input text
To begin with, status of a named entity could be one of these 3 probable values (with reference to this)
- history_status
- family_history_status
- probable_status
Describe the solution you'd like
CTakes leverages FSMs (finite state machines) for exactly this problem. However, any reliable algorithm can be used for this purpose. You can go through this PPT for better understanding of the working of CTakes pipeline that we are trying to adopt and improve. You can further read up here to gain more insights into the functionality of this processor.
This file can also be looked at to get more examples of what's expected from status context analysis.
Hi @Piyush13y , I had a look of related code StatusContextAnalyer.py in this ticket and information of CTake 4.0 NE+Context module. Just a clarification here as there are multiple "context" is mentioned in this ticket, and it seemed "we have implemented one part of the context analysis module i.e. [NegationContextAnalyzer] which detects negation of named entities. Now, we need the other aspect of it, StatusContextAnalyzer " , so it looks like we already have a wrapper(like) for CTake 4.0 NE+Context module. So about the new analyzer is it a standalone wrapper(like), or it is a part of the "contextAnalyer" and somehow belong to/re-use components of our (finished?) [NegationContextAnalyzer] ? (and also relationship to GitHub's CTakes NE context with this new analyzer). Thanks!
@J007X as of now, these two processors, namely StatusContextAnalyzer and NegationContextAnalyser will be mutually exclusive. Then, Based on what algorithm we leverage to implement Status analysis and if that and Negation analysis have a lot in common, we can maybe create a base context analysis class from implementation perspective, as future scope since as of now we may not have reusable components in Negation for Status analysis. But I would suggest, presently we focus on implementing a standalone StatusContextAnalyzer processor. Anyway, both Negation and Status will be standalone processors, individual components that can be plugged into our pipeline.
After some more research on how the CTake 4.0 code for handling StatusContext(analyzer) exactly dependent on, and it shows the FSMs(finite state machines) CTake currently using will be some non-trivial task to be ported (if need to be converted/translated/reuse-logic to python), as it have many dependencies -- for example the "StatusIndicatorFSM" is dependent on multiple java packages in ctakes.core.fsm and net.openai.util.fsm. So depends on situations it might not be practical or efficient to port them (than for example to call them directly) in some given time frame. Also there's no test cases in CTake source code so even if some partial porting to python is possible the resulting code will not very easy to verify. On the other hand if we want to use negex-like custom libraries (instead of CTale) we still need something like a paper on (exactly) how to implement the StatusContext Analyzer steps unlike that of the CTake-- just like the paper being used to write the current NegationContextAnalyzer
@J007X Thanks for the analysis. I think this task is much harder than expected, how about we try #10 to build a temporal analyzer. @Piyush13y will probably polish the description to provide more resources. But in the meantime, you could also see if you can find available models.