???? ??? ???? “????? ?? ??”? ?? ??? ?? ???. ?? ?? ??? ??? “????? ?? ?? 29,032?????”? ???. ???? ??? ???? ??? ?? ???? ???? ?? ????
?? ??(QA)? ?? ?? ??(IR)? ??? ??(NLP)? ??? ????? ????, ?? ??? ??? ??? ???? ??? ?? ??? ????. ?? ??? ???? ?? ??, ?? ?? ??????? ???, ? ??? ?? ?? ??? ??? ?? ?? ??? ?? ? ?? ?????. ? ??? ???? ?? ?? ??? ??? ?? ???? ?? ??? ???? ???.
?? ?? ??? ??? IR ??? ??? ??? ?? ?? NLP ???? ?????. ??? QA ??? ??? ??? ??? ????? ???? IR? NLP? ???? ?? ?? ?? ????. ??? NVIDIA Riva ? ??? QA ???? ?? ???? ?????.
NVIDIA Riva
NVIDIA Riva? ?? ? ?? ? ?? ?????? ???? ?? ??? ??? ?? GPU ?? SDK???. Riva SDK?? ??, ??? ??(NLP) ??? ???? ???? ??? ????. Riva? ??? AI ???? ?? ?????? ?? ?? ??? ?? ??? ??? ?? ??? ??? ??? ?? ??????. ?? ?????? Riva? QA ??? ?? ??? ??? ?? QA ??????? ??? ??? ?? ??? ???????.
Riva QA ??
Riva QA ??? ?? ??? ????? BERT(Bidirectional Encoder Representations from Transformers)?? ???? ????. BERT? 2018? ???? ??? ????? ??? NLP ?? ?? ????, NLP ??? ??? ??? ???. BERT? ????? ??? ??? ??? ??? ?????. ?????(Wikipedia)? ??? ??? ?? ???? ?? ?? ??????.
??? NLP ??? ?? ??? BERT? ???? SQuAD(Stanford Question Answering Dataset)? ?? ?? ??-?? ??? QA? ????? ?? ?? ??? ? ????. ?? ??? ??? ????(?? ?? ??)?? ???? ? ??? ?? ??? ?? ? ????. ?? 1? QA? ?? ?????. ??? “??? ???? ??? ?????”?? ??? ?? ???? “??”??? ??? ?? ??????. ? ??? ??? ?????? ????? ?? ??? QA ??? “??”??? ??? ???? ?????.
??: SQuAD: ???? ?? ??? ?? 100,000? ??? ??.
Riva? QA ??? ???
NVIDIA? ????? ??? ?? Riva? ?? ?? ??? ??? QA ??? ??????. Riva NLP ???? QA, NaturalQuery? ???? ?? API ?? ??? ?????. Wikipedia API ??? ???? ??? ????? ?????? ??? ??? ??? ??? ??? ? ????. ??? ?? ??? ?? ?? ??????. Wikipedia API ??? Riva QA ??? ???? ? ?? ???(Python) ??? ??? QA ???? ?? ? ????.
?? ??? ? Wikipedia API? ?????. ????, Riva? ?? ?? ?????? Riva NLP ??? API ? gRPC? ?????.
!pip install wikipedia
import wikipedia as wiki
import grpc
import riva_api.riva_nlp_pb2 as rnlp
import riva_api.riva_nlp_pb2_grpc as rnlp_srv?? ?? ??? ????. Wikipedia API ???? ?? ??? ???? ??? ?? ?? max_articles_combine?? ?????. “?? ???? ??????”?? ??? ? ?, ???? ??? ?? ??? ?????. ????? ? ??? ??? combined_summary ??? ?????.
input_query = "What is speech recognition?"
wiki_articles = wiki.search(input_query)
max_articles_combine = 3
combined_summary = ""
if len(wiki_articles) == 0:
print("ERROR: Could not find any matching results in Wikipedia.")
else:
for article in wiki_articles[:min(len(wiki_articles), max_articles_combine)]:
print(f"Getting summary for: {article}")
combined_summary += "\n" + wiki.summary(article)
????, Riva ??? ?? ?? ??? ???? gRPC ??? ???. Riva ??? ???? ?? ???? ‘localhost:50051’???. ?? ?? NaturalQueryRequest? ???????, ??? Riva ??? ???? ??? ????? ?? ?????. ????? Riva ???? ??? ??? ?????.
channel = grpc.insecure_channel('localhost:50051')
riva_nlp = rnlp_srv.RivaLanguageUnderstandingStub(channel)
req = rnlp.NaturalQueryRequest()
req.query = input_query
req.context = combined_summary
resp = riva_nlp.NaturalQuery(req)
print(f"Query: {input_query}")
print(f"Answer: {resp.results[0].answer}")
??
??? Riva QA ? Wikipedia API ???? ??? QA ??????? ??????. ?????, ?????? ??? ??? ??? ??? ??? ?? ? ????.
???? ??, ??, ?? ?? ?? ?? ?? ?? ??? ??? ?? ? ??????? ??? ??? ???. ?? ??? ??? ??? ?? ??? ?? ? ?? QA ???? ?? ? ????. ? ??? ??? Wikipedia API ??? ?? ???????? ?? ??? ???? IR ???? ?????? ?????? IR ???? ??? Riva? ?? ?? ? ??? ? ? ????.
Riva? ?? ??????? ??? ???!
