The Intersection between Linguistic Theories and Computational Linguistics over time
DOI:
https://doi.org/10.1590/1678-460X202238248453Palavras-chave:
Linguistics, Computational Linguistics, Natural Language Processing, Evolution of Computational LinguisticsResumo
Recent achievements have turned Computational linguistics into a dynamic research area, and an important field of application development that is being explored by leading technology companies. Despite the advances, there is still much room for improvement to allow humans to interact with computational devices, through natural language, in the same way that humans interact with native speakers of the same language. How to make computers understand or create new metaphors, metonymies or other figures of language? Is it possible to develop natural language systems capable of producing lyrics or poetry on the same level as humans? Can we produce systems capable of assigning new meanings to syntactic elements that are immediately perceived as coherent by humans? In this paper, we account for the evolution of computational linguistics, drawing a parallel with the evolution of linguistic theories and speculating about its research opportunities and foreseeable limitations.
Referências
Bates, M. (1995). Models of natural language understanding. In Proceedings of the National Academy of Sciences, 92(22),
-9982. http://doi.org/10.1073/pnas.92.22.9977
Baum, L. E., & Petrie, T. (1966). Statistical inference for probabilistic functions of finite state Markov chains. The Annals of Mathematical Statistics, v. 37, n. 6, p. 1554-1563. http://doi.org/10.1214/aoms/1177699147
Bellos, D. (2011). Is that a fish in your ear?: Translation and the meaning of everything. Penguin Books.
Bohannon, J. (2011, January 14). Google Books, Wikipedia, and the future of culturomics. Science, 331(6014), 135. http://doi.org/10.1126/science.331.6014.135
Bohn, D. (2019, January, 4). Amazon says 100 million Alexa devices have been sold. The Verge. https://www.theverge.com/2019/1/4/18168565/amazon-alexa-devices-how-many-sold-number-100-million-dave-limp (accessed November 22, 2021).
Bojanowski, P., Grave, E., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135-146. https://doi.org/10.1162/tacl_a_00051
Chomsky, N. (1957). Syntactic structures. Paris.
Dai, Z. et al. (2019). Transformer-xl: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp.
-2988). http://doi.org/10.18653/v1/P19-1285
Dalvi, F. et al. (2019). What is one grain of sand in the desert? analyzing individual neurons in deep NLP models. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence: AAAI 19, 31(01),
-6317. https://ojs.aaai.org/index.php/AAAI/issue/view/246 (accessed November 22, 2021).
Davies, M. (2009). The 385+ million-word Corpus of Contemporary American English (1990-2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14(2), 159-190. https://doi.org/10.1075/ijcl.14.2.02dav
Deng, L., & Liu, Y. (Ed.). (2018). Deep learning in natural language processing. Springer.
Devlin, J. et al. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1,
-4186. http://doi.org/10.18653/v1/N19-1423
Feldman, J. (2008). From molecule to metaphor: A neural theory of language. MIT press.
Fillmore, C. J. (1976). Frame semantics and the nature of language. In Annals of the New York Academy of Sciences: Conference on the Origin and Development of Language and Speech, 280, 20-32. https://www1.icsi.berkeley.edu/pubs/ai/framesemantics76.pdf (accessed November 22, 2021).
Firth, J. R. (1957). A synopsis of linguistic theory, 1930-1955. In J. R. Firth et al. (Eds.), Studies in Linguistic Analysis (pp. 1-32). Blackwell.
Griffiths, T. L. (2011). Rethinking language: How probabilities shape the words we use. In Proceedings of the National Academy of Sciences, 108(10),
-3826. https://doi.org/10.1073/iti1011108
Harris, R. A. (1995). The linguistics wars. Oxford University Press on Demand.
Heyman S. (2015). Google books: A complex and controversial experiment. The New York Times. https://www.nytimes.com/2015/10/29/arts/international/google-books-a-complex-and-controversial-experiment.html (accessed November 22, 2021).
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hutchins, W. J. (Ed.). (2000). Early years in machine translation: Memoirs and biographies of pioneers. John Benjamins Publishing.
Hutchins, J. (2003). ALPAC: the (in)famous report. In S. Nirenburg, H. L. Somers, & Y. A. Wilks (Eds.), Readings in machine translation (pp. 131-135). MIT Press. 1 https://doi.org/0.7551/mitpress/5779.001.0001
Jones, K. S. (1994). Natural language processing: a historical review. In A. Zampolli, N. Calzolari, & M. Palmer (Eds.), Current issues in computational linguistics: In honour of Don Walker (pp. 3-16). Springer, Dordrecht.
Jurafsky, D. M., & James, H. (2008). Speech and language processing: An introduction to speech recognition, computational linguistics, and natural language processing. Prentice Hall.
Kang, H., Yoo, S. J., & Han, D. (2012). Senti-lexicon and improved naïve Bayes algorithms for sentiment analysis of restaurant reviews. Expert Systems with Applications, 39 (5),
-6010. https://doi.org/10.1016/j.eswa.2011.11.107
Kepuska, V., & Bohouta, G. (2018). Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In 8th Annual Computing and Communication Workshop and Conference (pp. 99-103). IEEE. https://doi.org/10.1109/CCWC.2018.8301638
Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (pp. 1746-1751). https://doi.org/10.3115/v1/D14-1181
Koomey, J. et al. (2010). Implications of historical trends in the electrical efficiency of computing. In IEEE Annals of the History of Computing, 33(3), 46-54.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. In Communications of the Association of Computing Machinery, 60(6), 84-90. https://doi.org/10.1145/3065386
Kupiec, J. (1992). Robust part-of-speech tagging using a hidden Markov model. Computer Speech & Language, 6(3), 225-242. https://doi.org/10.1016/0885-2308(92)90019-Z
Kuzovkin, I. et al. (2018). Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex. Communications Biology, 1, 107. https://doi.org/10.1038/s42003-018-0110-y
Lafferty, J., Mccallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In C. E. Brodley, & A. P. Danyluk (Eds.), ICML ’01: Proceedings of the Eighteenth International Conference on Machine Learning (pp. 282-289). Morgan Kaufmann Publishers Inc. https://dl.acm.org/doi/proceedings/10.5555/645530 (accessed November 22, 2021).
Lakoff, G. (1963). Toward generative semantics. Technical report. UC Berkeley. https://escholarship.org/uc/item/64m2z2b1 (accessed November 22, 2021).
Lecun, Y. et al. (1998). Gradient-based learning applied to document recognition. In Proceedings of the IEEE, 86(11),
-2324. https://doi.org/10.1109/5.726791
Lee, J.-S., & Hsiang, J. (2020). Patent Claim Generation by Fine-Tuning OpenAI GPT-2. World Patent Information, 62, September 2020, 101983. https://arxiv.org/pdf/1907.02052.pdf (accessed November 22, 2021).
Levinson, S. E., Rabiner, L. R., & Sondhi, M. M. (1986). Hidden Markov model speech recognition arrangement (U.S. Patent n. 4, 587, 670). U.S. Patent and Trademark Office. https://patentimages.storage.googleapis.com/84/c0/08/af2eacbc2df545/US4587670.pdf (accessed November 22, 2021).
Luhn, H. P. (1958). The automatic creation of literature abstracts. IBM Journal of Research and Development, 2(2), 159-165. https://doi.org/10.1147/rd.22.0159
Macketanz, V., Burchardt, A., Uszkoreit, H. (2020). Tq-autotest: Novel Analytical Quality Measure Confirms That Deepl Is Better Than Google Translate. Technical report. The Globalization and Localization Association. https://www.dfki.de/fileadmin/user_upload/import/10174_TQ-AutoTest_Novel_analytical_quality_measure_confirms_that_DeepL_is_better_than_Google_Translate.pdf (accessed November 22, 2021).
Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. MIT press.
Mauchly, J. W. (1980). The ENIAC. In N. Metropolis, J. Howlett, & G.-C. Rota (Eds.), A History of Computing in the Twentieth Century (pp. 541-550). Academic Press.
Mccorduck, P. (1983). Introduction to the fifth generation. Communications of the ACM, 26(9), 629-630. https://doi.org/10.1145/358172.358177
Michel, J.-B. et al. (2011). Quantitative analysis of culture using millions of digitized books. Science, 331(6014), 176-182. https://doi.org/10.1126/science.1199644
Mikolov, T. et al. (2013). Efficient estimation of word representations in vector space. In International Conference on Learning Representations, Arizona. https://www.arxiv-vanity.com/papers/1301.3781/ (accessed November 22, 2021).
Minsky, M. (1974). A Framework for Representing Knowledge. Technical report. Massachusetts Institute of Technology. https://doi.org/10.1016/B978-1-4832-1446-7.50018-2
Morwal, S., Jahan, N., & Chopra, D. (2012). Named entity recognition using hidden Markov model (HMM). International Journal on Natural Language Computing 1(4), 15-23. https://doi.org/10.5121/ijnlc.2012.1402
Moto-oka, T., Stone, H. S. (1984). Fifth-generation computer systems: A Japanese project. Computer, 3, 6-13. https://doi.org/10.1109/MC.1984.1659076
National Research Council (US). (1966). Automatic Language Processing Advisory Committee (ALPAC). Language and machines: computers in translation and linguistics: A report. National Academies.
Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global vectors for word representation. In A. Moschitti, B. Pang, & W. Daelemans (Eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (pp. 1532-1543). Association for Computational Linguistics. https://doi.org/10.3115/v1/D14-1
Peters, M. E. et al. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1,
-2237. https://doi.org/10.18653/v1/N18-1202
Pollack A. (1992). ‘Fifth Generation’ Became Japan’s Lost Generation. The New York Times. https://www.nytimes.com/1992/06/05/business/fifth-generation-became-japan-s-lost-generation.html (accessed November 22, 2021).
Quillan, M. R. (1966). Semantic memory. Bolt, Beranak & Newman.
Radford, A. et al. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 9. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf (accessed November 22, 2021).
Rojas, R. (1997). Konrad Zuse’s legacy: the architecture of the Z1 and Z3. In IEEE Annals of the History of Computing, 19(2), 5-16. https://doi.org/10.1109/85.586067
Ruder, S. et al. (2019). Transfer learning in natural language processing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials (pp. 15-18). https://aclanthology.org/N19-5004.pdf (accessed November 22, 2021).
Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach. 3rd edition. Pearson Education Limited.
Schank, R. C. (1972). Conceptual dependency: A theory of natural language understanding. Cognitive Psychology, 3(4), 552-631. https://doi.org/10.1016/0010-0285(72)90022-9
Schank, R. C., & Abelson, R. P. (1975). Scripts, plans, and knowledge. In Proceedings of the Fourth International Joint Conference on Artificial Intelligence, 151-157. https://www.ijcai.org/Proceedings/75/Papers/021.pdf (accessed November 22, 2021).
Talmy, L. (2007). Attention phenomena. In D. Geeraerts, & H. Cuyckens (Eds.), The Oxford handbook of cognitive linguistics. Oxford university Press. https://doi.org/10.1093/oxfordhb/9780199738632.001.0001
Tavosanis, M. (2019). Valutazione umana di Google Traduttore e DeepL per le traduzioni di testi giornalistici dall’inglese verso l’italiano. In R. Bernardi, R. Navigli, & G. Semeraro (Eds.), CLiC-it 2019 - Proceedings of the Sixth Italian Conference on Computational Linguistics, Machine Translation, 2481, 494-525. http://ceur-ws.org/Vol-2481/paper70.pdf (accessed November 22, 2021).
Tenney, I, Das, D., & Pavlick, E. (2019). Bert rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp.
-4601
). Association for Computational Linguistics. https://arxiv.org/pdf/1905.05950.pdf (accessed November 22, 2021).
Thompson, C. A., Levy, R., & Manning, C. D. (2003). A generative model for semantic role labeling. In N. Lavrač et al. (Eds.) Machine Learning: 14th European Conference on Machine Learning Cavtat-Dubrovnik, Croatia (pp. 397-408). Springer-Verlag. https://link.springer.com/book/10.1007%2Fb13633 (accessed November 22, 2021).
Tulshan, A. S., & Dhage, S. N. (2018). Survey on Virtual Assistant: Google Assistant, Siri, Cortana, Alexa. In S. M. Thampi et al. (Eds.), International Symposium on Signal Processing and Intelligent Recognition Systems: 4th International Symposium SIRS 2018, Bangalore, India (p.p. 190-201). Springer. https://link.springer.com/book/10.1007/978-981-13-5758-9#toc (accessed November 22, 2021).
Turing, A. M. (1950). Computing machinery and intelligence. Mind, lix(236), 433-464. https://doi.org/10.1093/mind/LIX.236.433
Vaswani, A. et al. (2017). Attention is all you need. In U. von Luxburg, & I. Guyon (Eds.), NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems (pp.
-6010). Curran Associates Inc. https://dl.acm.org/doi/10.5555/3295222.3295349 (accessed November 22, 2021).
Viterbi, A. (1967). Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory, 13(2), 260-269. https://doi.org/10.1109/TIT.1967.1054010
Warren, D.H.D. (1982). A view of the Fifth Generation and its impact. AI Magazine, 3(4), 34-34. https://doi.org/10.1609/aimag.v3i4.380
Weaver, W. (1955). Translation. In W. N. Locke, & A. D. Booth (Eds.), Machine Translation of Languages: Fourteen Essays (pp. 15-23). MIT Press. https://repositorio.ul.pt/bitstream/10451/10945/2/ulfl155512_tm_2.pdf (accessed November 22, 2021).
Whorf, B. L., Carroll, J. B., & Chase, S. (Eds.) (1956). Language, thought, and reality: Selected writings of Benjamin Lee Whorf. MIT press.
Wittgenstein, L. (1953). Philosophical investigations. John Wiley & Sons.
Zipf, G. K. (1946). The psychology of language. In P. L. Harriman (Ed.), Encyclopedia of psychology (p.p. 332-341). Philosophical Library.
