- Iyunithi ye-ReLU yethula ukungalungi okudingekayo kumanethiwekhi e-neural ngokusebenzisa ukusebenza kwezibalo okulula nokuphumelelayo.
- Ikhono layo lokugwema ukugcwala kwe-gradient kokufakwayo okuhle lisheshisa ukuqeqeshwa uma kuqhathaniswa nemisebenzi efana ne-sigmoid noma i-tanh.
- Kunezinhlobo eziningana ezifana ne-Leaky ReLU noma i-GELU eyenzelwe ukunciphisa inkinga yama-neuron angasebenzi noma ukuthuthukisa ukushelela kwe-curve.
- Ucwaningo lwamuva lubonisa ukuthi, ngemikhawulo ethile, i-ReLU ingenza imisebenzi edlula eminye ekubonisweni kwemizwa okungacacile.

Uma uke wangena ezweni lokufunda okujulile, cishe uke wahlangana neyunithi eqondile elungisiwe , eyaziwa kangcono ngokuthi i-ReLU. Uma uqala ukubuka, kubonakala sengathi umsebenzi ulula kakhulu ukuba ube namandla kangaka, kodwa yilokho kanye okulula okushintshe indlela esiqeqesha ngayo amanethiwekhi e-convolutional neural kanye nama-transformer anamuhla.
Empeleni, i-ReLU isebenza njengesihlungi esivumela amanani amahle ukuthi adlule angaxutshwanga futhi aguqule noma iyiphi inombolo engemihle ibe yi-zero. Leli khono lokwethula ukungalingani ngaphandle kokusebenzisa izibalo eziyinkimbinkimbi yilona elivumela i-AI ukuthi ifunde amaphethini ayinkimbinkimbi nagobile ngaphandle kokuthi ikhompyutha iphahlazeke enkambisweni.
Isebenza kanjani ngempela i-ReLU?
Ngokwezibalo, sikhuluma ngomsebenzi ochazwa ngokuthi f(x) = max(0, x). Ngamagama alula, lokhu kusho ukuthi uma idatha yokufaka inkulu kuno-zero, umphumela uyidatha efanayo; uma ingaphansi noma ilingana no-zero, umphumela ungu-0. Le ndlela yokwenza yiyo enikeza amanethiwekhi e-neural ukuguquguquka kokwenza imodeli yemisebenzi engeyona eye-linear , okubalulekile ukuze imodeli ingabi nje ukuhlehla okukhulu okuqondile.
Ngokombono wokusebenza, i-ReLU iyigugu ngoba ishibhile kakhulu ukubala i- . Ngokungafani neminye imisebenzi edinga ama-exponential, lapha kukhona ukuqhathanisa okulula kuphela. Ngaphezu kwalokho, i-derivative yayo ilula kakhulu: ingu-1 ngamanani amahle kanye no-0 ngamanani angalungile. Nakuba ngokweqiniso i-derivative ingekho ku-zero, izinhlaka eziningi ezifana ne-PyTorch zimane ziyinike inani elingu-0 ukuze zigweme izinkinga.
Izinzuzo ezibalulekile kanye nesimo sokuhlakazeka

Enye yezinzuzo ezinkulu ukuthi ayihlushwa ukugcwala lapho okufakwayo kukuhle. Emisebenzini emidala njenge-sigmoid, ama-gradients aba mancane kakhulu, okwenza izendlalelo zokuqala zenethiwekhi ziyeke ukufunda. Nge-ReLU, i-gradient ihlala ingaguquki ku-1, okulwa nokufiphala kwe-gradient futhi kuvumela amanethiwekhi okuqeqesha anezendlalelo eziningi.
Elinye iphuzu elithakazelisayo ukusebenza okusabalele. Njengoba i-ReLU isetha kabusha konke okufakwayo okungekuhle kube yi-zero, nganoma yisiphi isikhathi ingxenye yama-neurons kuphela esebenzayo . Lokhu kulingisa kakhulu ukusebenza kobuchopho futhi kwenza imodeli isebenze kahle, njengoba kungewona wonke amayunithi akhipha izimpawu ngasikhathi sinye.
Inkinga yama-neurons "afile"
Akusikho konke ukukhanya kwelanga nama-rose, njengoba kunengozi yama -ReLU angasebenzi noma afe . Lokhu kwenzeka lapho i-neuron ihlala ithola amanani angalungile kuyo yonke idatha yokuqeqesha; njengoba i-gradient yayo ingu-zero, isisindo asishintshi, futhi i-neuron iba yinto engenamsebenzi. Ngokuvamile, lokhu kwenzeka ngenxa yokusebenzisa izinga lokufunda eliphakeme kakhulu , okusunduza ukuthambekela kwe-neuron ngaphansi kuka-zero.
Ukuvimbela imodeli ukuthi ingabi yisigamu esingenalutho, amasu anjengo -He initialization , olungisa izisindo zokuqala, noma i-Batch Normalization, esiza ukugcina ukusebenza kungaphakathi kwamazinga anempilo, avame ukusetshenziswa.
Izinhlobo zokuthuthukisa ukusebenza
Ukuze kuxazululwe ukushiyeka kwe-ReLU ejwayelekile, kuye kwavela ezinye izindlela eziningana ezithakazelisayo:
- I-ReLU Evuzayo: Esikhundleni se-absolute zero kuma-negative, isebenzisa i-slope encane kakhulu (njengo-0,01) ukuze i-neuron iqhubeke nokufunda okuthile.
- ISINGENISO: Kufana nokwangaphambilini, kodwa ukuthambekela okungekuhle akukalungiswa, kodwa kunalokho imodeli iyakufunda lokho ngesikhathi sokuqeqeshwa.
- I-RELU6: Ikhawulela umphumela ophezulu ku-6, okuwusizo kakhulu kumadivayisi eselula noma ihadiwe ene- izibalo ezinembile kakhulu.
- I-ELU ne-GELU: Lezi zici zihlose ukwenza ijika libe bushelelezi. I-GELU, isibonelo, iyinkanyezi kumamodeli afana no-BERT, kusukela ihlanganisa izakhiwo ze-stochastic futhi isebenza kangcono emisebenzini yolimi kanye neyombono.
Ukuqhathaniswa ne-Sigmoid ne-Tanh
Ngaphambi kokuba i-ReLU ithandwe ngenxa ye-AlexNet ngo-2012, indlela ejwayelekile kwakuwukusebenzisa ama-sigmoid noma ama-hyperbolic tangent (tanh) curves. Inkinga ukuthi womabili ayathamba emaphethelweni, okwenza ukuqeqeshwa kuhambe kancane kakhulu. Eqinisweni, kuye kwaboniswa ukuthi inethiwekhi esebenzisa i-ReLU ingafinyelela iphutha layo lokuqeqesha ngokushesha kakhulu kuneyo esebenzisa i-tanh. Noma kunjalo, i-sigmoid curve isalokhu iyisinqumo esithandwayo sezingqimba zokukhipha lapho kudingeka khona amathuba.
I-ReLU iphambili: I-INR kanye ne-BW-ReLU

Muva nje, kube nempikiswano mayelana nokuthi i-ReLU iyafaneleka yini kuma-Implicit Neural Representation (ama-INR), okuyinethiwekhi efunda izimpawu eziqhubekayo njengezithombe ze-3D noma ukuskena kwezokwelapha. I-ReLU kwakucatshangwa ukuthi inobandlululo lwe-spectral , okusho ukuthi yayinenkinga yokufunda imininingwane emincane noma evame kakhulu.
Kodwa-ke, kuvele isiphakamiso esibizwa ngokuthi i-BW-ReLU . Lo mklamo awusebenzisi ama-ReLU ngamanye, kodwa kunalokho uwahlanganisa ukuze akhe imisebenzi ye-B-spline efana ne-wavelet. Ngokukhawulela ama-neurons ngale ndlela, kufezwa ukulungiswa okungcono , okuvumela inethiwekhi ukuthi ifunde imininingwane emincane futhi idlule nemisebenzi ekhethekile njenge-SIREN emisebenzini enesisombululo esikhulu noma ekwakheni kabusha isithombe sezokwelapha.
Ukusetshenziswa kwe-ReLU kuhlala kuyinketho ekhethwayo lapho isivinini kanye nokuhambisana kwehadiwe kubaluleke kakhulu. Ku-PyTorch, isibonelo, ukusebenzisa inketho ethi `inplace=True` kukuvumela ukuthi ubhale ngaphezulu i-input tensor, ulondoloze i-RAM—imininingwane ebalulekile lapho usebenza namamodeli amakhulu.
Lo msebenzi wokwenza kusebenze uguquke kusukela ekuqhathanisweni okulula okungu-zero waba yitshe lesisekelo lokufunda okujulile, okwenza kube lula ukuqeqeshwa kwamanethiwekhi ajulile ngenxa yokusebenza kahle kwawo kanye nokususwa kokugcwala. Kusukela ezinguqulweni zawo zakudala kuya ezakhiweni ezintsha ze-BW-ReLU, ikhono lawo lokuzivumelanisa nezimo nokuguquka liqinisekisa ukuthi lihlala liyithuluzi eliyisisekelo lokucubungula ulwazi ku-AI yanamuhla.
Imibuzo ebuzwa njalo
Yini ngempela eyenziwa umsebenzi wokwenza kusebenze i-ReLU?
Lo msebenzi ubuyisela inani lokufaka uma lihle futhi ubuyisela u-zero uma okufakwayo ku-negative noma u-zero.
Uyini umehluko phakathi kwe-ReLU ne-Leaky ReLU?
Nakuba i-ReLU iguqula zonke iziphetho ezimbi zibe yi-zero, i-Leaky ReLU ivumela ingxenye encane yamanani angalungile ukuthi idlule ngokusebenzisa i-slope eqondile.
Iyini inkinga nge-ReLU efayo?
Lokhu kwenzeka lapho i-neuron ihlala ithola ama-input angalungile futhi i-gradient yayo iba yi-zero, ivimbela izisindo zayo ukuthi zingabuyekezwa futhi ishiye ingasebenzi unomphela.
Kungani i-ReLU ingcono kunomsebenzi we-sigmoid ezingqimbeni ezifihliwe?
I-ReLU igwema inkinga yokunyamalala kwe-gradient ngoba ayigcwalisi amanani amahle, okusheshisa kakhulu ukuqeqeshwa kwenethiwekhi.
Iyiphi i-ReLU6 variant esetshenziselwa yona?
Le nguqulo ikhawulela ukukhishwa okuphezulu ku-6 ukuze kuthuthukiswe ukumelwa kwedatha ezinhlelweni ezisebenzisa izibalo ezinembile, njengamadivayisi eselula.
Kukuziphi izimo lapho i-sigmoid noma i-tanh zisasetshenziswa khona?
Zisetshenziswa kakhulu ezingqimbeni zokukhipha ukuze kukhiqizwe amathuba noma emasangweni eseli aphindaphindayo lapho kudingeka khona isikhawu esilinganiselwe.
Iyini i-BW-ReLU futhi isetshenziselwani?
Kuyisakhiwo esihlanganisa ama-neurons e-ReLU ukuze alingise ama-wavelet e-B-spline, okuthuthukisa ukufunda kwemininingwane yemvamisa ephezulu ekumeleleni kwemizwa okungabonakali.